
CampbellCollaboration 


Education Coordinating Group 


Employment Interventions for Return to Work in 
Working Aged Adults Following Traumatic Brain 
Injury (TBI): A Systematic Review 


Carolyn W. Graham, Michael D. West, Jessica L. Bourdon, 
Katherine J. Inge and Hannah E. Seward 




CampbellCollaboration 


The Campbell Library comprises: 

• Systematic reviews (titles, protocols and reviews) 

• Policies and Guidelines 

• Methods Series 

Go to the library to download these resources, at: 
www.campbellcollaboration.org/librarv 


Better Evidence for a Better World 


Colophon 


Title 

Employment Interventions for Return to Work in Working-Aged Adults Following 
Traumatic Brain Injury (TBI): A Systematic Review 

Institution 

The Campbell Collaboration 

Authors 

Graham, Carolyn W. 

West, Michael D. 

Bourdon, Jessica L. 

Inge, Katherine J. 

Seward, Hannah E. 

DOI 

io.4073/csr.20i6.6 

No. of pages 

133 

Citation 

Graham CW, West MD, Bourdon JL, Inge KJ. Employment Interventions for Return to 
Work in Working Aged Adults Following Traumatic Brain Injury (TBI): A Systematic 
Review. Campbell Systematic Reviews 2016:6. DOI: io.4073/csr.20i6.6 

ISSN 

1891-1803 

Copyright 

© Graham et al. 

This is an open-access article distributed under the terms of the Creative Commons 

Attribution License, which permits unrestricted use, distribution, 

and reproduction in any medium, provided the original author and source are credited. 

Roles and 
responsibilities 

Carolyn W. Graham and Michael D. West contributed to the writing of the report and 
revising of this protocol. Katherine J. Inge contributed to the writing of the report and 
revising of this protocol. Jessica L. Bourdon contributed to the search and revising of this 
protocol and writing of the report. Hannah Seward contributed to the writing of the 
report. 

Editors for 

this review 

Editor: Sandra Jo Wilson 

Managing editor: Carlton J. Fong 

Sources of support 

This study was supported by the Center on Knowledge Translation for Employment 
Research through grant 90DP0009 (formerly ED #Hi33Aiooo26) to SEDL, an affiliate 
of American Institutes for Research, from the National Institute on Disability, 

Independent Living, and Rehabilitation Research (NIDILRR) in the U.S. Department of 
Health and Human Services (HHS), Administration for Community Living. However, the 
contents of this article do not necessarily represent the policy of the U.S. Department of 
HHS, and you should not assume endorsement by the Federal Government. 

Declarations of 

interest 

Carolyn Graham, Jessica Bourdon, and Jessica Seward have no vested interest in the 
outcomes of this review, nor any incentive to represent findings in a biased manner. 
Michael D. West and Katherine J. Inge have published peer-reviewed articles on 
vocational rehabilitation interventions, including research articles concerning supported 
employment interventions. 

Corresponding 

author 

Carolyn W. Graham 

Department of Physical Medicine & Rehabilitation 

Virginia Commonwealth University 

PO Box 843038 

Richmond, VA 23237 

ii 

The Campbell Collaboration www.campbellcollaboration.org 


USA 

E-mail: Carolyn . W. Graham (5> vcuhealth . or g 

Full list of author information is available at the end of the article 


The Campbell Collaboration | www.campbellcollaboration.org 


Campbell Systematic Reviews 


Editor-in-Chief 

Julia Littell, Bryn Mawr College, USA 

Editors 


Crime and Justice 

David B. Wilson, George Mason University, USA 

Education 

Sandra Wilson, Vanderbilt University, USA 

International 

Development 

Birte Snilstveit, 3ie, UK 

Hugh Waddington, 3ie, UK 

Social Welfare 

Nick Huband, Institute of Mental Health, University of Nottingham, UK 
Geraldine Macdonald, Queen’s University, UK & Cochrane Developmental, 
Psychosocial and Learning Problems Group 

Methods 

Therese Pigott, Loyola University, USA 

Emily Tanner-Smith, Vanderbilt University, USA 

Chief Executive 

Officer 

Howard White, The Campbell Collaboration 

Managing Editor 

Karianne Thune Hammerstrom, The Campbell Collaboration 

Co -Chairs 


Crime and Justice 

David B. Wilson, George Mason University, USA 

Peter Neyroud, University of Cambridge, UK 

Education 

Sarah Miller, Queen’s University Belfast, UK 

Gary W. Ritter, University of Arkansas, USA 

Social Welfare 

Brandy R. Maynard, Saint Louis University, USA 

Mairead Furlong, National University of Ireland, Maynooth, Ireland 

International 

Development 

Peter Tugwell, University of Ottawa, Canada 

Hugh Waddington, 3ie, India 

Methods 

Ian Shemilt, EPPI-Centre, UK 

Ariel Aloe, University of Iowa, USA 


The Campbell Collaboration was founded on the principle that systematic 
reviews on the effects of interventions will inform and help improve policy 
and services. Campbell offers editorial and methodological support to review 
authors throughout the process of producing a systematic review. A number 
of Campbell’s editors, librarians, methodologists and external peer-reviewers 
contribute. 

The Campbell Collaboration 

PO Box 4404 Nydalen 

0403 Oslo, Norway 
www.camDbellcollaboration.or2 


IV 


The Campbell Collaboration | www.campbellcollaboration.org 


Table of Contents 


SYNOPSIS/PLAIN LANGUAGE SUMMARY 7 

EXECUTIVE SUMMARY/ABSTRACT 10 

Background 10 

Objectives 10 

Search Methods 10 

Selection Criteria 11 

Data Collection and Analysis 11 

Results 11 

Authors’ Conclusions 12 

1 BACKGROUND 13 

1.1 The Problem, Condition, or Issue 13 

1.2 The Intervention 15 

1.3 How the Intervention Might Work 18 

1.4 Why it is Important to do the Review 19 

2 OBJECTIVES 22 

3 METHODS 23 

3.1 Criteria for Considering Studies for This Review 23 

3.2 Search Methods for Identification of Studies 25 

3.3 Data Collection and Analysis 26 

4 RESULTS 29 

4.1 Description of Studies 29 

4.2 Risk of Bias in Included Studies 38 

4.3 Synthesis of Results 39 

4.4 Summary of Main Results 41 

4.5 Overall Completeness and Applicability of Evidence 44 

4.6 Quality of the Evidence 45 

4.7 Limitations and Potential Biases in the Review Process 46 

4.8 Agreements and Disagreements With Other Studies or Reviews 47 


5 


The Campbell Collaboration | www.campbellcollaboration.org 


5 AUTHORS’ CONCLUSIONS 49 

5.1 Implications for Practice and Policy 49 

5.2 Implications for Research 50 

6 REFERENCES 52 

6.1 References to Included Studies 52 

6.2 References to Excluded Studies 52 

6.3 hand-searched References 59 

6.4 Additional References 61 

7 INFORMATION ABOUT THIS REVIEW 71 

7.1 Review Authors 71 

7.2 Roles and Responsibilities 73 

7.3 Sources of Support 73 

7.4 Declarations of Interest 73 

7.5 Plans for Updating the REview 74 

7.6 Author Declaration 74 

8 TABLES 75 

8.1 Table 1: Time Periods of Employment Outcomes Reported by Study 75 

8.2 Table 2: Search Strings used for First Search 76 

8.3 TAble 3: Characteristics of Included Studies 77 

8.4 Table 4: Characteristics of Excluded Studies 79 

8.5 Table 5: Risk of Bias of Included Studies 84 

8.6 Table 6: Group Results 85 

9 DATA AND ANALYSES 86 


6 


The Campbell Collaboration | www.campbellcollaboration.org 



Synopsis/Plain Language Summary 


The Campbell review in brief 

Programs for adults who have suffered traumatic brain injury may improve 
employment status. However, no programme was found to be more effective than its 
comparator programme. 

What is this review about? 

The unemployment rate in the United States for people who have suffered traumatic 
brain injury (TBI) was around 60 percent for 2001-10, compared to a national average 
of around 7 percent. Post-acute rehabilitation services - such as residential community 
reintegration programs, comprehensive day treatment programs, and community re- 
entry programs - focus on helping individuals adjust to ongoing impairments and to re- 
enter their communities, workplaces, and education. This review assesses the most 
effective type of intervention for returning individuals with TBI to work. 

What is the aim of this review? 

This Campbell systematic review examines the effectiveness of vocational rehabilitation 
interventions to help adults with traumatic brain injury get competitive employment. 
Three randomized controlled trials are included: two of military populations in the 
United States and two of the civilian population in China (Hong Kong). 

What studies are included in this review? 

Studies are included which assess interventions focused on assisting helping working- 
aged adults with TBI return to competitive employment, including self-employment. 
Participants must have been between 18 and 65 years of age, experienced a non- 
penetrating TBI, been engaged in either full-time or part-time employment at time of 
injury, and been unemployed or on medical leave at time of receipt of the intervention. 
The studies must have competitive employment as an outcome. 

Three randomized controlled trials (RCT) are included in the analysis, two studying US 
military personnel and one of civilians in China (Hong Kong). All three studies 
compared alternative programmes. One compared an intensive in-hospital program 
versus an at-home program. The second study compared the CogSMART program plus 
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supported employment with supported employment alone. And in the third study the 
control group received psycho-educational training with the treatment group receiving 
the same content via a virtual reality-based training platform. 

How effective were the programs? 

None of the programs were better at improving employment outcomes than the 
comparator program to which it was compared. That is the intensive in-hospital 
program was no better than the at-home alternative, CogSMART added to no value to 
supported employment alone, and virtual reality-based training was no better than 
psycho-educational training. Comparison of employment before and after the 
interventions suggests that the interventions in the United States improved 
employment status, but not that in China 

None of the studies reported secondary employment outcomes: hours worked and 
wages earned. 

It was not possible to conduct analysis of the relative effectiveness of different types of 
programme because of the small number of included studies. 

What are the implications of this review for policy makers and decision 
makers? 

These three studies have limited implications for practice and policy. No intervention 
was found to be more effective than any other. In two of the studies the populations 
were limited to military subjects, who present with significantly different challenges 
such as posttraumatic stress disorder. 

The comparator interventions of at-home training and supported employment appear 
promising. 

What are the research implications of this review? 

There is a need for more RCTs on RTW interventions with adults with TBI, preferably 
separating competitive employment from school attendance. Military interventions 
should be conducted with civilian samples in order to determine their effectiveness in 
the civilian population, including on populations outside the United States. A broader 
range of employment outcomes should be studied, with regular follow up at standard 
intervals (e.g., six months, 12 months, 18 months, etc.). 

How up-to-date is this review? 

The search was completed in 2015. This Campbell Systematic Review was published in 
July 2016. 

What is the Campbell Collaboration? 


The Campbell Collaboration | www.campbellcollaboration.org 



The Campbell Collaboration is an international, voluntary, non-profit research network 
that publishes systematic reviews. We summarise and evaluate the quality of evidence 
about programmes in the social and behavioural sciences. Our aim is to help people 
make better choices and better policy decisions. 

About this summary 

This summary was written by Howard White (Campbell Collaboration). This PLS is 
based on Campbell Systematic Review 2016:6 ‘Employment Interventions for Return to 
Work in Working Aged Adults Following Traumatic Brain Injury (TBI): A Systematic 
Review’ by Carolyn W. Graham, Michael D. West, Jessica L. Bourdon, Katherine J. Inge 
and Hannah E. Seward (DOI:io.4073/csr.20i6.6). Anne Mellbye (RBUP, Norway) 
designed the summary, which was edited and produced by Tanya Kristiansen (Campbell 
Collaboration). 
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Executive Summary/ Abstract 


BACKGROUND 


Individuals with traumatic brain injury (TBI) often struggle to obtain competitive 
employment after sustaining a TBI, commonly as a result of the post-injury difficulties 
they exhibit (Andelic, Stevens, Sigurdardottir, Arango-Lasprilla, & Roe, 2009; 

Mansfield et al., 2015). The currently reported unemployment rate for people with TBI 
is approximately 60% (Cuthbert et al., 2015). Hence, the unemployment for individuals 
with TBI is considerably higher than for individuals without disabilities. 

Many adults with TBI seek assistance in gaining employment through post-acute 
rehabilitation. Post-acute rehabilitation services focus on helping individuals adjust to 
ongoing impairments and to re-enter their communities, workplaces, and education. 
Post-acute interventions are broadly classified into the following groups (Shames, 
Treger, Ring, & Giaquinto, 2007): residential community reintegration programs, 
comprehensive day treatment programs, and community re-entry programs that focus 
on vocational and social reintegration. This review focused on identifying the most 
effective type of intervention for returning individuals with TBI to work. The authors of 
this review drew on a wide range of databases, searched grey literature, included studies 
with a range of follow-up times, and focused on competitive employment outcomes 
among individuals with TBI regardless of the injury severity of the individuals. 


OBJECTIVES 


This review examined the effectiveness of vocational rehabilitation (VR) interventions 
to help adults with TBI attain competitive employment. 


SEARCH METHODS 


Studies for this review were identified by searching 16 databases, including the 
Australian Education Index, CIRRIE - the Center for International Rehabilitation 
Research Information and Exchange Databases, the Academic Complete collection, 
EBSCOhost Research Databases, MEDLINE/PubMed, ProQuest, and 11 other 
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databases. Unpublished papers and grey literature were also searched. Reference lists of 
papers included in the analysis and previous systematic reviews were searched. 


SELECTION CRITERIA 


The following inclusion criteria were used for each potential study: (l) Studies are of 
interventions focused on assisting helping working-aged adults with TBI return to 
competitive employment, including self-employment. (2) Competitive employment had 
to be measured as an outcome. (3) Participants must have been between 18 and 65 years 
of age, experienced a non-penetrating TBI, been engaged in either full-time or part-time 
employment at time of injury, and been unemployed or on medical leave at time of 
receipt of the intervention. Studies that included individuals with other disabilities were 
included only if the results were provided for TBI participants separately from those 
with other disabilities. (4) Data had to be presented separately for competitively 
employed participants. (5) The design must be a randomized controlled trial (RCTs) or 
quasi-experimental design, with a treatment and a control/comparison group. 


DATA COLLECTION AND ANALYSIS 


Odds ratios and log odds ratios were computed and 95% confidence intervals were 
computed for each included study. Only datab for the primary outcome, competitive 
employment status, were used due to insufficient secondary outcome data. 


RESULTS 


The literature search resulted in 6,941 unduplicated documents. From these 6,941 
documents, 414 documents were selected for full-text review. From these 414 
documents, 67 intervention reports were found, with only three return-to-work RCTs 
meeting inclusion criteria for this systematic review. The last search was conducted 
November 7, 2015. 

The three studies included in this review were RCTs with parallel interventions (Man et 
al., 2013; Salazar et al., 2000; Twamley et al., 2014, 2015). Man et al. (2013) compared 
two interventions, an artificial intelligent virtual reality-based training program (n = 17) 
and a psycho-educational vocational training system (n = 20) using a civilian 
population from China. Salazar et al. (2000) compared an in-hospital cognitive 
treatment (n = 67) to an in-home treatment (n = 53) for active U.S. military personnel. 
Twamley et al. (2014, 2015) compared a CogSMART Plus supported employment 
program (n = 21) to an enhanced supported employment program (n = 21) for U.S 
military veterans. 
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Salazar and colleagues (2000) used active military samples and Twamley et al. (2014, 
2015) used U.S. military veterans. Man and colleagues (2013) used civilians in 
China.The sample in Salazar et al. (2000) was predominantly African American and 
White. There were more Hispanic/Latino and White participants in the Twamley et al. 
(2014, 2015) study. Man and colleagues (2013) RCT did not report the ethnicity or race 
of its sample from China. The Twamley et al. (2014, 2015) and Man et al. (2013) studies 
reported severity of injury as mild to moderate. 


All studies were RCTs; however, blinding, incomplete data, and selective outcome 
reporting were of concern for all studies. Design, review status, publication type, and 
presence of control group were sufficient.Findings from this systematic review were 
inconclusive in that all odds ratios were not significant. Man et al. (2013) had the largest 
odds ratio (OR = 2.204, P = 0.264) but had the lowest employment rates (30%). Salazar 
et al. (2000) and Twamley et al. (2014, 2015) had odds ratios less than one. This 
indicated that the alternate interventions for Salazar et al. (OR = 0.514, p = 0.353) and 
Twamley et al. (OR = 0.817, p = 0.749) were more effective than the primary 
intervention. The employment rates for Salazar et al. and Twamley et al. ranged from 
55% to 94%, which were rates higher than observed in Man et al. In sum, there were no 
significant odds ratios. Although all interventions evidenced positive average gains, no 
intervention was identified as more effective than another. 


AUTHORS’ CONCLUSIONS 


All interventions showed positive average gains. However, we were unable to determine 
which intervention was most effective due to the small number of studies (n = 3). More 
experimental RCTs need to be conducted with interventions not included in this study. 
There are several recommendations for the direction of research concerning return-to- 
work for adults with TBI. First, studies of return-to-work VR interventions for adults 
with TBI must improve their quality of research by conducting RCTs. Second, 
separating competitive employment from education would provide a more accurate 
estimate of impact on return to work. Third, research is needed with other populations 
outside the United States and civilian samples. Last, future return-to-work VR studies 
should report time to employment, hours worked, separate rates of competitive 
employment, sheltered employment, educational training, and continued follow-ups at 
12 months or more. 
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l Background 


l.i THE PROBLEM, CONDITION, OR ISSUE 


1.1.1 Incidence and Causes of Traumatic Brain Injury 

Traumatic brain injury (TBI) occurs when an external force causes an alteration to brain 
function (e.g., loss of consciousness, memory loss, and neurologic defects). A TBI can 
occur from a blow to the head, blast waves from an explosion, swift acceleration or 
deceleration, or the penetration of a foreign object into the brain (Maas, Stocchetti, & 
Bullock, 2008; Menon, Schwab, Wright, & Maas, 2010). TBI is a major concern 
worldwide and affects 10 million people annually (Hyder, Wunderlich, Puvanachandra, 
Gururaj, & Kobsingye, 2007). The two most common worldwide causes of TBI are 
traffic accidents and falls, with traffic accidents declining in predominance and falls 
rising due to aging populations (Brazinova et al., 2015; Li et al., 2015; Majdan, Rusnak, 
Brazinova, & Mauritz, 2015; Peeters et al., 2015; Popescu, Anghelescu, Daia, & Onose, 
2015; Scholten, Haagsma, Panneman, Beeck, & Polinder, 2014; Shekhar, Gupta, 
Premsagar, Sinha, & Kishore, 2015; Shivaji, Lee, Dougall, McMillan, & Stark, 2014). 

1.1.2 Severity and Sequelae 

Levels of TBI range from mild to severe and are determined by measures such as 
duration of coma or post-traumatic amnesia (PTA), Glasgow Coma Scale (GCS) scores, 
and the nature and extent of functional impairments following the injury. Individuals 
with TBI experience various combinations of physical, cognitive, sensory, and emotional 
symptoms, which are largely determined by the severity of the impact, the location of 
the impact, and whether the TBI is a penetrating (open) or non-penetrating (closed) 
injury (BrainandSpinalCord.org, 2013). 

Brain injury severity is typically related to the degree and extent of impairment post- 
injury. Even individuals experiencing mild brain injuries can experience problems in 
one or more areas of functioning, though most individuals with a mild TBI will recover 
lost functioning. In most cases of moderate and severe TBI, individuals’ recovery will 
plateau below pre-injury functioning levels. 
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1.1.3 Return to Work Following TBI 

Individuals with TBI often struggle to either obtain or maintain competitive 
employment after sustaining a TBI, commonly as a result of the post-injury difficulties 
they exhibit (Andelic, Stevens, Sigurdardottir, Arango-Lasprilla, & Roe, 2009; Dikmen 
et al., 1994; Ezrachi, Ben-Yishay, Diller, & Rattock, 1991; Mansfield et al., 2015; Ruttan, 
Martin, Liu, Colella, & Green, 2008). Reported unemployment rates of people with TBI 
have ranged from 45% to 78% (Cuthbert et al., 2015; Doctor et al., 2005; Kendall, 
Muenchberger, & Gee, 2006; Yasuda, Wehman, Targett, Cifu, & West, 2001). This 
variation in return-to-work (RTW) rates is related to differences in sample 
characteristics, methodologies, and measures of employment. Nevertheless, 
unemployment for individuals with TBI is considerably higher than for individuals 
without disabilities. 

TBI and its effects can be overwhelming, not only for individuals with this injury and 
their families (Cifu, Craig, & Rowland, 1996; Liu, Zhu, Liu, & Guo, 2015; Moriarty et al., 
2015; Tam, McKay, Sloan, & Ponsford, 2015), but also for other sectors of society. While 
individuals and families experience financial burdens due to the loss of jobs and wages, 
as well as medical and rehabilitation expenses (Costa, Dagher, Lamoureux, Guise, & 
Feyz, 2015; Fadyl & McPherson, 2009; Joseph et al., 2015; Ma, Chan, & Carruthers, 
2014; Samuelsson, Tropp, & Lundqvist, 2014; Te Ao et al., 2014), communities and 
employers experience lost workdays and productivity (Te Ao et al., 2014; Andlin- 
Sobocki, Jonsson, Wittchen, & Olesen, 2005; Samuelsson et al., 2014). TBI can also 
impose substantial burdens on governments through publicly funded healthcare, 
rehabilitation service costs, and financial support for injured persons (Te Ao et al., 

2014; Hyder et al., 2007; Samuelsson et al., 2014). The Centers for Disease Control and 
Prevention (CDC) in the United States, estimated that the medical and indirect costs of 
TBI (e.g., lost productivity) in 2010 were $76.5 billion in the United States (Coronado, 
McGuire, Faul, Sugerman, & Pearson, 2012; Finkelstein, Corso, & Miller, 2006), while 
the total annual cost in Europe was approximately €386 billion (Andlin-Sobocki, 
Jonsson, Wittchen, & Olesenet al., 2005). Thus, the cost of TBI to both the individual 
and society is staggering. These costs can be ameliorated through rehabilitation and by 
returning individuals with TBI to work (Doctor et al., 2005; Kendall et al., 2006; 
Samuelsson et al., 2014; Yasuda et al., 2001). 

Shames, Treger, Ring, and Giaquinto (2007) conducted a comprehensive review of the 
literature regarding factors associated with successful RTW following TBI and current 
rehabilitation strategies. They note that predicting a successful RTW involves a complex 
interaction of variables, including pre-morbid factors (e.g., age, education, and prior 
employment history), injury-related factors (type, severity, and location), post-injury 
impairments, and personal and environment factors (e.g., marital status, alternative 
income sources, and social support). Because of this complexity and the unique 
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characteristics of each individual with a TBI, prediction of RTW is not feasible. They 
note, however, that the literature includes numerous examples of high-risk individuals 
who have been successful when given sufficient rehabilitation and support. In addition, 
the TBI employment research literature is international (e.g., Cancelliere et al., 2014; 
Fear et al., 2008; Li et al., 2015; Shekhar et al., 2015; Te Ao et al., 2014), which brings 
into consideration differences across economies, cultures, health care and rehabilitation 
systems, and national landscapes of public and social disability policies. 

Despite the variety of factors related to poor employment outcomes for individuals with 
TBI, some factors are more frequently reported to contribute to poor employment 
outcomes (Shames et al., 2007): 

• More severe injury (Cuthbert et al., 2015; Kreutzer et al., 2003; Trexler, Trexler, 
Malec, Klyce, & Parrott, 2010; Wagner, Hammond, Sasser, & Wiercisiewski, 
2002), as measured by the GCS, PTA, duration of rehabilitation, or other means 

• Higher age at injury (Cuthbert et al., 2015; Keyser-Marcus et al., 2002; Kreutzer 
et al., 2003) 

• Gender (Female) (Cuthbert et al., 2015) 

• Lower levels of pre-injury educational and/or occupational status (Cuthbert et 
al., 2015; Hart, Whyte, Polansky, Kersey-Matusiak, & Fidler-Sheppard, 2005; 
Keyser-Marcus et al., 2002; Walker, Marwitz, Kreutzer, Hart, &Novack, 2006) 

• Limited social support from friends, family, neighbors, church, etc. (Yasuda et 
al., 2001) 

• Significant physical, psychosocial, or cognitive impairments (Cuthbert et al., 
2015; Keyser-Marcus et al., 2002; McCrimmon & Oddy, 2006; Wagner et al., 
2002; Walker et al., 2006) 

• Member of a minority group (Arango-Lasprilla et al., 2008; Cuthbert et al., 

2015; Hart et al., 2005; Kreutzer et al., 2003) 

• History of substance abuse (Bogner, Corrigan, Mysiw, Clinchot, & Fugate, 2001; 
Corrigan, 1995; Hollar, McAweeney, & Moore, 2008; McAweeney, Jones, & 
Moore, 2008; Wagner et al., 2002) 

• Financial status and receipt of a settlement or public benefits following the 
injury (Cuthbert et al., 2015) 


1.2 THE INTERVENTION 


1.2.1 Post-Acute Return-to-Work Interventions 

Following acute care, post-acute rehabilitation services focus on helping individuals 
adjust to ongoing impairments and re-enter their communities, workplaces, schools, 
etc. Post-acute interventions are broadly classified into the following groups (Shames et 
al., 2007): (1) Residential community reintegration programs that provide intensive 
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behavioral interventions; (2) Comprehensive day treatment programs that emphasize 
training in self-awareness, social skills, daily living skills, and coping mechanisms; and 
(3) Community re-entry programs that focus on vocational and social reintegration. 

Within these, RTW interventions are funded and delivered through multiple avenues. In 
the United States, the primary sources of RTW interventions are employee disability 
management programs and state vocational rehabilitation (VR) agencies. Many other 
countries have equivalent organizations (United Nations World Health Organization & 
The World Bank, 2011). 

Brief descriptions of employee disability management and VR programs are presented 
in this section. 

Employee Disability Management Programs, typically provided directly by employers 
or through their insurance programs, are a combination of many services (e.g., work 
injury prevention programs, early intervention, transitional work programs, and case 
management) to better address the physical and psychological needs of the worker 
(Ongori, 2012; Shrey, 1996). Job accommodations, support services, and determining 
the use of adaptive devices are all potential components of disability management 
(Ongori, 2012; Shrey, 1996). For example, gradual RTW would fall into this category. 
Additionally, some companies may offer on-site training as a disability management 
component. This may be similar to transitional programs, where the client receives 
physical, cognitive, and behavioral training while developing job skills in a practice 
work environment. This allows employers to see how and if they need to make 
accommodations, and helps the employee develop the skills they need to perform on the 
job (Ben-Yishay et al., 1985; Jacobs, 1997; Klonoff et al., 2007; Prigatano et al., 1994). 

State VR Services are provided to individuals with various types of disabilities with the 
goal of increasing their independence and employment. Services include physical 
restoration (prosthetic devices), vocational training, counseling, and job development 
assistance. Supported employment is an increasingly utilized VR service option for 
clients with TBI, where a client obtains paid work that is integrated with people without 
disabilities while receiving ongoing support from an employment specialist or job coach 
(Wehman et al., 2003). An unique feature of the supported employment model is that 
individuals with TBI receive training on the job as opposed to pre-training. 

An individual placement model of supported employment consists of an employment 
specialist or job coach who assists the individual with TBI in finding a job. This 
specialist then helps coordinate pre-employment supports (e.g., benefits counseling or 
transportation) and may provide on-the-job support and training (Wehman et al., 

2003), in addition to other counseling or advocacy services that are needed (Wehman et 
al., 1993). These supports provided by the employment specialist vary from person to 
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person and change over time. Eventually, the specialist’s involvement reduces to 
periodic checks, as mandated by the Rehabilitation Act (U.S. Department of Education, 

2004) . These checks are critical to long-term support and the ongoing employment of 
individuals with TBI. Such long-term support is unique to the supported employment 
model (Wehman et al., 2003). 

Job accommodations are common to the generally differing systems of Employee 
Disability management programs and state VR services. Title I of the Americans with 
Disabilities Act (ADA; Americans with Disabilities, 2008) requires employers with 15 or 
more employees to consider providing reasonable accommodation for employees who 
meet the ADA’s definition of disability. Accommodations are negotiated between the 
employee and the employer and are determined on a case-by-case basis. Before 
becoming eligible for a reasonable accommodation, the applicant or employee with TBI 
must disclose his or her disability status to the employer and make a request for the 
accommodation. 

Accommodations are alterations made for the employee that do not conflict with the 
responsibilities of the job, such as flexible scheduling to accommodate the person”s 
disabilities (e.g., epilepsy, chronic fatigue, or psychosocial disabilities). Environmental 
modifications are a type of job accommodation and include building ramps for 
wheelchair access, adding elevators, relocating an employee’s office, and widening 
aisles. With the increasing use and availability of technology such as high-speed 
Internet, some businesses permit telework from home, allowing an employee to 
accomplish their daily tasks without having to leave their residence (West & Anderson, 

2005) . In addition, employers may also reassign non-essential job functions that the 
employee cannot perform due to his or her disability. Employers are not required to 
make accommodations for essential job functions by eliminating them from the job 
description. Essential functions are the fundamental, crucial job duties performed in a 
position. 

Another accommodation strategy common to both systems is to provide assistive 
technology services and devices. These can include memory aids, touch-screen 
computers, adjustable mounts at desks, specialized computer software, or alternative 
input devices to allow a person with limited manual dexterity or fine motor skills to use 
a computer (Inge &Targett, 2007). With the advent of handheld devices such as 
Smartphones and tablet computers, individuals can use pre-installed programs to create 
schedules and set alarms (Gentry, Lau, Molinelli, Fallen, & Kriner, 2012). There are 
Applications (apps) for these devices that may be used to train employees with 
disabilities such as TBI by providing visual cues for how to complete tasks, ways to deal 
with interpersonal situations, and individualized task lists (Burke, Andersen, Bowen, 
Howard, & Allen, 2010; Chang, Chen, Chuang, 2011; Gentry et al., 2012; Tsai, 2012). 
Since Individuals with TBI often have memory deficits, the use of handheld devices with 
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apps is an example of an accommodation that can facilitate skill acquisition and job 
retention. 

Finally, both service systems can offer ancillary therapeutic services which, though not 
directly employment-focused, support the RTW process. These include physical 
therapy, occupational therapy, speech-language therapy, neuro-cognitive therapy, and 
peer or professional counseling. Utilizing these programs as part of a cohesive 
rehabilitation strategy emphasizes a holistic approach (Ben-Yishay et al., 1985; 
Prigatano et al., 1984, 1994). 


1.3 HOW THE INTERVENTION MIGHT WORK 


1.3.1 Supply-Side and Demand-Side Interventions 

Employment barriers and interventions may be broadly grouped into either supply-side 
or demand-side sources (Chan et al., 2010). Supply-side barriers are those related to 
individuals with disabilities, such as skill deficits, interfering behaviors, functional 
limitations related to the disability, and accommodation needs. Vocational 
rehabilitation provides support to individuals with disabilities on the supply-side of the 
labor market by providing training and placement services to job seekers (Luecking, 
2008). The intent of these services is to prepare individuals with disabilities for 
employment and then supply these trained workers to employers. Supply-side 
interventions typically include the assessment of skill deficits followed by skill training 
and the provision of accommodations (including assistive technology) to prepare the job 
seeker for employment. Supported employment is a supply-side intervention. However, 
this is a “place and then train” approach to provide a labor supply to employers 
(Wehman, Inge, Revell, & Brooke et al., 2007). Individuals with TBI are supported in 
identifying their preferences for employment, and then are presented to potential 
employers for meeting labor market needs (Wehman & Targett, 2006). An employment 
specialist or job coach provides training and support on the job until the worker with 
disabilities performs to the satisfaction of the workplace. VR counselors, employment 
specialists, therapists, and educators typically deliver supply-side interventions. The 
duration of these interventions can range from days to years depending on the nature of 
the services. For example, an individual receiving supported employment services 
should receive ongoing support throughout his or her work history. 

Conversely, demand-side interventions are increasingly seen as important for assisting 
individuals with disabilities in becoming employed (Chan et al., 2010; Gilbride & 
Stensrud, 1999; Luecking, 2008; Wehman, Kregel, & Brooke, 2008). Examples of 
demand-side barriers include employers’ limited experience with employees with 
disabilities, employer attitudes toward hiring individuals with disabilities, high turnover 
or absentee rates, high production demands, and high costs related to injured workers. 
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Demand-side interventions may include disability management practices such as 
gradual return to full duty following injury, employer training and technical assistance 
related to disability and accommodation, co-worker and supervisor support and 
assistance, and employer-provided internships, skills training, or support programs. 

The Logic Model (see Figure l) presents the supply-side and demand-side intervention 
activities, recipients of intervention activities, and short- and long-term outcomes. Both 
types of interventions for individuals with TBI are included. 

Figure l. Logic Model, Employment Interventions for RTW in Working Aged Adults 
After TBI 


Situation Outputs Outcomes 


Activities Participants Short-Term Long-Term 


Supply-Side Interventions 


o Traumatic brain injury (TBI) 
often results in changes to 
cognitive abilities, physical 
functioning, somatic 
functioning, and psychological 
/ behavioral functioning, 
o These can in turn lead to poor 
outcomes, particularly RTW 
or school. 

o Young adults are at risk for 
TBI. 


•Skills training 
•Therapies 
•Service and support 
strategies 

• Prosthetics / orthotics 
•Workplace 
accommodations 
•Other adaptive or 
assistive technology 
devices 


o Employment rates following 
TBI are low, especially for 
those with more serious 
injuries. 

o Inability to RTW following TBI 
can lead to other family, 
social, and psychological 
problems. 


• Employer training • 

regarding disability • 

• Employer /supervisor 

training and support • 
•Co-worker support 
•Assistance with job 
accommodations • 

• Disability management 

practices 

• Modifications to 

policies and 
procedures that 
discriminate against 
job-seekers or 
employees with 
disabilities 


Job-seekers or 
employees with 
TBI 

Family members of 
individuals with TBI 


s Increased skills, 
knowledge, and 
abilities of 
individuals with 
TBI 

v' Increased 
functional 
abilities 
v' Increased 
prospects for 
employment, re- 
employment, or 
RTW 


v' Increased 
employment of 
individuals with 
TBI and other 
types of 
disabilities 
v' Decreased 
dependence on 
public assistance 
and support from 
friends and 
family members 


Demand-Side Interventions 


Business owners 
Human Resources 
staff 

Supervisors of 
employees with 
disabilities 
Co-workers of 
employees with 
TBI 


V Increased 
willingness and 
capacity to 
recruit, hire, train, 
and retain 
individuals with 
TBI 

s Increased skills 
and knowledge 
regarding 
accommodating 
employees with 
TBI 

s Socialization and 
relationships 
between 
employees with 
and without 
disabilities 


V Lower costs to 
employers 
related to 
disability and 
injured workers 
v' Lower costs to 
taxpayers 
related to 
disabled and 
injured workers 
S More workplace 
diversity and 
disability-friendly 
businesses 


1.4 WHY IT IS IMPORTANT TO DO THE REVIEW 
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Previous TBI systematic reviews have not compared the efficacy of different types of VR 
interventions on competitive employment. Instead, they have examined the 
effectiveness of specific interventions for individuals with TBI regarding their 
community integration (Kim & Colantonio, 2010); cognitive rehabilitation (Cicerone et 
al., 2011; Rohling, Faust, Beverly, & Demakis, 2009); quality of life (Berger, Leven, 
Pirente, Bouillon, & Neugebauer, 1999); functional independence (Willemse-van Son, 
Ribbers, Verhagen, & Stam, 2007); community participation (Cattelani, Zettin, & 
Zoccolotti, 2010; Evans & Brewis, 2008); and physical, psychological, and social 
functioning (Carney et al., 1999; Cattelani et al., 2010; Hellweg & Johannes, 2008; 
Kennedy et al., 2008). 

There also have been systematic reviews that have examined the effectiveness of 
interventions at improving vocational outcomes for those with acquired brain injury 
(ABI) (Turner-Stokes, Nair, Sedki, Disler, & Wade, 2005). Specifically, Van Velzen, Van 
Bennekom, Edelarr, Sluiter, and Frings-Dresen (2009) examined the number of people 
with ABI— a diagnosis that includes brain injury from non-traumatic causes such as 
stroke, disease, and substance abuse— who returned to work over a 16-year period, but 
did not look at VR interventions. Similarly, Nightingale, Soo, and Tate (2007) 
conducted a systematic review that examined the prognostic factors related to people 
with TBI and RTW. Two of the outcome variables were productivity and competitive 
employment, but Nightingale et al. (2007) did not examine interventions. Additionally, 
a recent Campbell Collaboration title registration by Braathen and colleagues (2011) 
proposed to examine the effectiveness of comprehensive occupational rehabilitation 
programs for RTW among people on long-term sickness absence. Their review will 
include people with mental disorders and/or musculoskeletal disorders, but not those 
with TBI. While necessary, This further illustrates the need for the current systematic 
review. Saltychev, Eskola, Tenovuo, & Laimi (2013) conducted a review on predictive 
factors for employment after TBI, which included interventions. This review included 
studies that examined individuals that had worked or studied before injury. 

Further, Fadyl and McPherson (2009) reviewed and evaluated the evidence for the 
effectiveness, strengths, and weaknesses of vocational interventions for individuals with 
TBI. Unlike the longer time period (1973 to 2015) and broad information retrieval 
(more databases and grey papers) of this proposed review, Fadyl and McPherson 
limited their search to articles available in English from January 1990 to July 2007. 
Further, their search was restricted to the Ovid Journals database, which includes 
MEDLINE, Psyclnfo, CINAHL, AMED, health and psychosocial instruments, evidence- 
based medicine databases, and Web of Science. Geurtsen, Van Heugten, Martina, and 
Geurts (2010) conducted a systematic review of the effectiveness of comprehensive 
rehabilitation program interventions for a number of outcome variables, including 
employment, only PubMed, PsycINFO, and PsychLit databases and articles published 
from 1990 to 2008 were searched. Similarly, a recent systematic review protocol that 
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addresses the effectiveness of multidisciplinary post-acute rehabilitation for adults with 
moderate to severe TBI (Agency for Healthcare Research Quality, 2011) limited its 
electronic database search to MEDLINE, the Cochrane Central Register of Controlled 
Trials, PEDro, and PsycINFO. 

Unlike previous systematic reviews, this review focused on identifying the type of 
intervention that was most effective in returning individuals with TBI to work. This 
review searched a wider range of databases, searched and included grey literature, 
searched a broader length of time (1973 to 2011), and focused on competitive 
employment outcomes among persons with TBI regardless of injury severity. By 
including all competitive employment, supply- and demand-side interventions, and 
evidence-based research interventions for persons with TBI, this systematic review may 
provide some information concerning the effectiveness of these employment 
intervention programs that will guide practitioners and VR policymakers. Further, this 
review provides information concerning the direction for future research. 
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2 Objectives 


This review examined the effectiveness of interventions on competitive employment 
outcomes for adults with TBI. 
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3 Methods 


The methods for this systematic review are based on the protocol published by the 
Campbell Collaboration Library of Systematic Reviews (Graham & West, 2014). The 
protocol can be accessed at http : / / www. campbellcollaboration. org /lib / project/ 241 / 


3.1 CRITERIA FOR CONSIDERING STUDIES FOR THIS 
REVIEW 


3.1.1 Types of Studies 

A priori criteria were developed for the selection of studies to be included in this review. 
The following criteria were used. Each study must have: (1) explored interventions that 
assist working-aged adults with TBI to return to competitive employment (with 
competitive employment as the outcome), including self-employment; (2) at least 
included the primary outcome and employment status, and may also have included 
secondary outcomes, length of time employed, and mean hours worked; (3) participants 
who were between the ages of 18 and 65, with a non-penetrating TBI, engaged in either 
full-time or part-time employment at the time of injury, and unemployed or on medical 
leave at the time of receipt of intervention (participants could have been injured on or 
off the job); and (4) presented data separately for competitive (i.e., open or 
independent) employment and the non-competitively employed participants or 
students. Studies that included individuals with other disabilities were included only if 
the results were provided for TBI participants separate from those with other 
disabilities. The designs of the included studies could be randomized controlled trials 
(RCTs) or quasi-experimental designs, with a treatment group and a 
control/ comparison group consisting of treatment as usual, other appropriate 
interventions, or no intervention. 

Types of Participants 

Participants were between the ages of 18 and 65 years of age, had a non-penetrating 
TBI, engaged in either full-time or part-time employment at the time of injury, were 
unemployed or on medical leave at the time of receipt of intervention, and were injured 
either on or off the job. All included studies consisted of only participants with TBI. No 
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subsets of TBI participants were used. Penetrating TBI was not included because these 
injuries have a more complex sequelae and require more extensive rehabilitation than 
non-penetrating injuries. 

Types of Interventions 

To be included, studies must have used interventions that helped working-aged adults 
with TBI to return to competitive employment (including self-employment), with 
competitive employment as the outcome. The intensity of the intervention could vary. 
All intervention types— such as transitional, supported, and holistic strategies— were 
included. 

Types of Outcome Measures 

Primary Outcomes 

The primary RTW outcome was a dichotomous variable, competitive employment 
status, in which individuals with TBI were either employed full-time or part-time for 45 
days or more or not employed. The premise for using competitive employment as the 
one outcome variable is that competitive employment represents optimal real-world 
functioning. The definition of competitive employment was based on three criteria: 

(1) Employed individuals with TBI perform employment-related tasks in an integrated 
setting (i.e., not a disability organization) and must interact with co-workers who are 
predominantly non-disabled. 

(2) Employed individuals with TBI are hired and paid by the business where the work is 
performed, not by a disability organization (such as facility-based program or sheltered 
workshop). In this way, competitively employed individuals with TBI have the same 
risks of losing their position as non-disabled individuals. 

(3) Employed individuals with TBI are also paid commensurate with that received by 
non-disabled co-workers who have similar qualifications and perform similar duties. 
When an individual with TBI is competitively employed, the individual vied for a 
position against individuals without disabilities, which is more difficult than sheltered 
employment or volunteer work, and, in most cases, more difficult than entering an 
educational program. In contrast, individuals who volunteer or are employed in facility- 
based programs typically work at lower performance or production standards than 
those required in competitive employment. Opportunities to re-do their performance 
through retaking exams or courses are not typically available in competitive work 
environments. 

Secondary Outcomes 
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Based on the protocol (Graham & West, 2014), expected secondary outcomes were 
length of time to competitive employment and hours worked. However, none of the 
studies included reported this information. Thus, no secondary outcomes were 
analyzed. 


3.2 SEARCH METHODS FOR IDENTIFICATION OF STUDIES 


3.2.1 Language and Geographic Origin 

The search was restricted to published or unpublished research written in English or 
Spanish and published between 1973 and 2015. To include reports in all languages 
would be untenable and fiscally burdensome to include in this report. 

3.2.2 Electronic Searches 

Two trained research assistants conducted the literature search, sixteen databases (with 
125 sub-search engines within them) were divided between the two research assistants. 
Each research assistant searched one database at a time, using appropriate search terms 
(see Table 2) for the database, before moving on to the next assigned database. One 
research assistant was assigned to conduct grey literature searches not related to 
databases. 

A spreadsheet was constructed and shared with authors and research assistants to 
maintain a record of searches. Each database had a separate tab in which the name of 
the searcher, date, database, search string, year, first author, journal/source, title, and 
DOI were entered. Each day, research assistants reviewed titles of documents located 
during the search that appeared to describe an employment intervention. If research 
assistants could not determine relevance from the title, they read the abstract. These 
relevant abstracts and all search results were saved as PDFs by the research assistants 
and uploaded to a shared electronic folder, as well as printed out for record-keeping. 
Two authors (Graham and West) reviewed each relevant abstract and requested the full 
article for those that appeared appropriate for the systematic review. If documents were 
not available online, research assistants acquired copies from the university library or 
submitted an interlibrary loan request. These documents were subsequently added to 
the shared online folder for review. 

Studies for this review were identified by searching 16 databases: Australia Education 
Index/Australian Council for Educational Research, CIRRIE - Center for International 
Rehabilitation Research Information and Exchange Database, www.ClinicalTrials.gov, 
Cochrane Central Registry of Controlled Trials, Ebrary: Academic Complete Collection, 
EBSCOhost Research Database, FirstSearch, Google Scholar, MEDLINE/PubMed, 
NARIC REHABDATA, ProQuest, PsycEXTRA, PsycINFO, Science Direct, Web of 
Knowledge, and Web of Science. Within FirstSearch, nine sub-search engines were 
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searched. Within ProQuest, 47 sub-search engines were searched. Within EBSCOhost, 
69 sub-search engines were searched. Further, unpublished papers and grey literature 
were also searched. Reference lists of papers included in the analysis and previous 
systematic reviews concerning RTW employment and TBI were also searched. 

Grey papers were also searched via 67 national and state websites, which included eight 
international websites: 

• Great Britain 

o 

o https:/ /www.gov.uk/ financial-help-disabled/ overview 

o 

• European Union 

• http://www.epr.eu/ 

o http://ec.europa.eu/social/main.isp?catId=i57&:langId=en 

• Australia 

o http://www.crsaustralia.gov.au/ 
o http://deewr.gov.au 

• European Union Grey Papers 

• www.opengrey.eu 


3.3 DATA COLLECTION AND ANALYSIS 


3.3.1 Selection of Studies 

Three team members (Graham, West, & Inge) independently reviewed the amassed 
abstracts of articles and reports selected by the research assistants as potentially eligible 
for the systematic review. Articles were then screened by title and abstract. If the title 
and abstract suggested that the article might meet the inclusion criteria, the article was 
assigned to two of the three authors for review. Each author was assigned a set of 
articles to review, which included coding and determining whether the study should be 
included. Each reviewer used the approved coding template, in which characteristics of 
the study were coded, such as type and description of intervention, number of subjects, 
group and subject characteristics, and data needed for analysis. (See Appendix or 
Graham and West, 2014, for coding template.) reviews stopped if the reviewer 
determined that the article did not meet the inclusion criteria. After both assigned 
authors had reviewed the article, the authors met to discuss the article, coding, and 


26 


The Campbell Collaboration | www.campbellcollaboration.org 


inclusion decision. Each article was discussed concerning coding. If there was a 
difference in coding, the authors discussed the coding and came to a consensus, 
inclusion required both assigned reviewers to come to a consensus on coding and the 
inclusion decision. A consensus of included studies for this systematic review was 
reached without requiring a third reviewer to resolve selection conflicts. None of the 
authors were involved in any of the studies reviewed, and a third-party reviewer was not 
needed to minimize bias. 

3.3.2 Data Extraction and Management 

After coding included studies, inter-rater reliability was established above 85%. Using 
coded forms for each included study, the authors entered data into Comprehensive 
Meta-Analysis 2.2 software. 

Assessment of Risk of Bias in Included Studies 

Cochrane’s risk of bias tool (i.e., sequence generation, allocation concealment, blinding, 
incomplete data, and selective outcome reporting; Higgins & Altman, 2008), along with 
study design, review status, type of publication, and control group, were used to assess 
the chance of bias. Each factor was coded as low risk, high risk, or unclear/unknown for 
each potential study to be included, “unclear” means insufficient detail was reported in 
the study, what was done in the study was unknown and the risk of bias is therefore 
unknown, or the risk of bias item was irrelevant to the study (Higgins & Altman, 2008). 
This latter point is especially relevant with regard to blinding and incomplete data on 
the outcome variable (Higgins & Altman, 2008). For more information concerning the 
high risk, low risk, or unclear ratings, see the coding form in the Appendix. In addition, 
design, review process, publication type, and presence of a control group (Higgins & 
Altman, 2008) were coded for quality. This was completed at the time of coding, 
included in the coding template, and discussed by the two assigned reviewers. 

Data from the three identified random control studies were entered into Comprehensive 
Meta-Analysis 2.2 software (CMA) from coding sheets. All three studies compared two 
interventions. Each study was entered into the CMA program. As the protocol states 
(Graham & West, 2014), multiple time points for employment status (event) reported 
within a single study were averaged, and the average employment status was the datum 
used. No meta-analysis was performed (as stated in the protocol) because of the small 
number of studies. Odds ratios and standard error for log odds ratios were computed, 
and a forest plot of the effect sizes was constructed, as stated in the protocol (Graham & 
West, 2014). 

Unit of Analysis Issues 

The unit of analysis for this synthesis was at the group level, since each study had two 
viable and independent intervention groups. 
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Dealing With Missing Data 

Attrition occurred in all groups. The total number used in the calculations was the total 
number in the intervention at the last time point. A description of missing data is 
provided in the results section. 

Assessment of Heterogeneity 

As stated in the protocol (Graham & West, 2014), we had planned to assess 
heterogeneity by interpreting the statistical significance of Qs and the value of I 2 . The 
criteria for determining heterogeneity were a statistically significant Q (p < 0.05) and a 
moderate I 2 of 0.50 (Borenstein, Hedges, Higgins, & Rothstein, 2009). Due to the small 
number of studies identified, a meta-analysis was not performed and the assessment of 
heterogeneity was not needed. 

Assessment of Reporting Biases 

As stated in the protocol (Graham & West, 2014) we planned to use a funnel plot to 
assess publication bias (Biostat, 2005). This was not performed due to the small 
number of included studies. 

Subgroup Analysis 

As stated in the published protocol (Graham & West, 2014), we planned to conduct 
eight subgroup comparisons: (1) level of education pre-injury; (2) severity of TBI; (3) 
type of intervention; (4) provider (demand- or supply-side); (5) duration of intervention 
using six-month increments (six months or less, 6.1 months to 12 months, etc.) 1 ; (6) 
subject population (civilian or military); (7) where the injury occurred (fall, motor 
vehicle/traffic accident, colliding with an object, assault, combat/terror); and (8) type of 
comparison group (no intervention, treatment as usual, other intervention). These were 
not performed due to the small number of studies and lack of information. 

Group Comparisons 

Group comparisons were not performed due to the small number of studies. 

Sensitivity Analysis 

Sensitivity analysis was not performed, as mentioned in the protocol (Graham & West, 
2014). 
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4 Results 


4.1 DESCRIPTION OF STUDIES 


4.1.1 Results of the Search 


An extensive literature search was completed on VR and RTW of adults with TBI (see 
Figure 2). 


511,471 possible documents 
were identified through 
databases. 


193,874 grey literature documents 
were found. 



414 documents were read. 



21 studies were found through a 

hand search of references. 

< 



^ 6,527 documents were excluded. 


67 documents were assessed for 
eligibility. 



64 documents were excluded. 


Figure 2. Specific overview of article selection. 


The literature search utilized 16 search engines and 125 sub-search engines to search for 
published, peer-reviewed articles and grey literature. In addition, 67 websites were 
searched for grey literature. This yielded a total of 705,362 studies, which included 
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duplicates (511,471 possible documents within the search engines 193,874 documents 
examined on Internet websites with grey literature and 21 additional articles found 
through hand searching past systematic reviews and primary study reference lists). In 
order To manage data, a series of Google Drive folders were constructed. The “All 
Search Documents” folder contained a folder for each search engine. Each of these 
search engine folders consisted ofcontained the actual search results for each term used, 
research assistants who were assigned search engines kept track of the number of 
documents found in the “Unduplicated Results” folder. After each search term was run 
in a search engine, the research assistant would determine the overall number of hits, 
upload the search hits, screen each hit by article title of the article to determine whether 
further examination was needed by other team members, and determine whether there 
were duplicates from past searches. Of those 705,362 documents, 6,941 were 
unduplicated. The document title and abstract were read to determine whether 
documents were relevant to the systematic review. Unpublished articles with titles and 
abstracts that were considered potentially eligible were recorded in the “All 
GreyDocument” spreadsheet file. The record for a potential grey document consisted 
contained of name of the research assistant, date found, website address, year the 
document was published, author, title, type of document (e.g., dissertation, 
presentation, etc.), search string that located the document, and note (access status). 
When the document was available, a copy of the document was uploaded into the “Grey 
PDFs” folder. Once the document had been read by a researcher, the document would 
be either moved to the “Print Out” folder for further review or to the “Viewed” folder. 
Researchers discussed which studies should be examined further. The identified 
documents were then printed out for further review. If it was decided that the study 
might fit the criteria, researchers coded the document. Documents in the “Viewed” 
folder were those found not to meet the inclusion criteria. Peer-reviewed articles were 
processed similarly. From these 6,941 documents, the abstracts of 374 peer-reviewed 
articles and 40 grey literature documents were reviewed. Of the 414 peer-reviewed and 
grey documents examined, 67 presented findings on the effectiveness of interventions 
for RTW for working-aged adults with TBI and were assessed for eligibility. The last 
search was conducted on November 7, 2015. 

4.1.2 Included Studies 

Of the 67 intervention study reports identified, only three studies (Man, Poon, & Lam, 
2013; Salazar et al., 2000; Twamley, Jak, Delis, Bondi, & Lohr 2014, and Twamley et al. 
2015) fit all of the inclusion criteria. Note that Twamley reported outcomes in two 
articles. These two articles are being counted as one article, as both articles concern the 
same study. All three RCTs included in this review used other interventions that were 
suspected to be less effective than the “treatment” group. Below, both the primary 
treatment and alternate treatment groups are described. 
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The Man et al. (2013) study was a RCT study conducted in China with a civilian sample. 
This study was conducted with 50 individuals who had mild to moderate TBI and were 
recruited from Hong Kong hospitals and rehabilitation facilities. Man et al. did not 
report The cause of TBI. Inclusion criteria required participants to have a loss of 
consciousness of less than six hours. There was no report of mean length of loss of 
consciousness for either group. However, the Glasgow Coma Scale was used to assess 
the severity of TBI, and the mean score on this scale was 10.25 (SD = 1.51; Range = 9- 
14) for the artificial intelligent virtual reality-based training program (AIVTS; treatment 
group) and 10.05 (SD = 1.31; Range= 9-13) for the psycho-educational vocational 
training system (PEVTS; second intervention). No time since injury information was 
reported. 

In the Man et al. study (2013), 50 participants were randomly assigned to either the first 
intervention group (AIVST) or the second intervention group (PEVTS) (Man et al., 
2013). Each group began with 25 participants. Both groups had similar informational 
content but the training platform differed by group. The AIVST group received 12 
sessions using a 3-D artificial intelligence reality-based training software program. The 
program begins the training by determining the skill level of the participants. As the 
participant progresses through training, the program can upgrade, maintain, or 
downgrade the training level to meet the training needs of the participant based on 
performance. Therefore, when participants performed well, the program would increase 
difficulty. If the participant’s performance declined, the program would revert to 
previous levels of training. 

The PEVTS group also received 12 training sessions with similar problem-solving tasks, 
content, and structure, given by a vocational trainer. The trainer provided instruction, 
time to practice skills, and games on problem solving specified by the manual. The 
fidelity of the administration of this intervention was assessed by four trained 
rehabilitation professionals. The fidelity agreement intra-class coefficient was 0.645, P 
= 0.021. 

Although the study authors referred readers to a demographic table, no demographic 
table was published in the article. Forty participants (20 per group) provided three 
follow-up reports of employment outcome at one, three, and six months. Primary 
outcomes consisted of Tower of London assessment, Wisconsin Cord Sorting Test, 
Vocational Cognitive Rating Scale, and employment outcomes. Employment outcomes 
were measured by four categories: full-time employment, supported employment, 
sheltered employment, and unemployed/unable to return to work. No other 
employment measures were reported. 

Of the 20 participants in the AIVTS group (treatment group) that were assessed at 
follow-ups, 17 (85%) were unemployed and three (15%) were employed at one month; 
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14 (70%) were unemployed and six (30%) were employed at three months; and 12 
(60%) were unemployed and eight (40%) were employed at six months. Of the 20 
participants in the PEVTS group that were assessed at follow-ups, 20 (100%) were 
unemployed and zero were employed at one month; 16 (80%) were unemployed and 
four (40%) were employed at three months; and 16 (80%) were unemployed and four 
(40%) were employed at six months. The odds ratio was 2.204 (95% confidence interval 
[Cl] = 0.459, 10.573). 

The Salazar et al. (2000) study was conducted on a United States military sample and is 
based on three articles that used the same dataset from one RCT with parallel group 
interventions: (1) an intensive in-hospital, cognitive treatment group, and (2) an in- 
home program without a VR group (Braverman et al., 1999; Salazar et al., 2000; 

Warden et al., 2000). The Salazar et al. (2000) article was the complete report that met 
all the inclusion criteria for this systematic review and was used in this review. 

The Salazar et al. (2000) study was an RCT with parallel treatment groups, conducted 
with 120 active duty military service persons who had acquired a TBI. There is some 
discrepancy in the reporting of the severity of injury. Salazar et al. report that the 
injuries were “moderate to severe” (p. 3035), but state later that the injuries were 
“relatively severe” based on “the rates of axonal shear injury on MRI (92% and 95%), 
cerebral contusions (51% and 54%), posttraumatic amnesia of 7 days or more (41% to 
42%), and traumatic unconsciousness for 24 hours or more (30% and 38%), 
respectively, in the hospital and home treatment groups” (p. 3077). Another article from 
this study reported “moderate to severe TBI” (p. 1100) based on “all patients reach[ing] 
the inclusion criteria of Rancho los Amigos level of 7” (Warden et al., 2000, p. 1097). 
Most participants obtained TBI through motor vehicle/traffic incident, assault, and 
alcohol-related accidents. The mean loss of consciousness was 35 hours. All participants 
were within 90 days of injury. Participants in the study were randomly assigned to one 
of the two treatment groups. Each group had a separate treatment team from the other, 
and the treatment teams worked independently. There were periodic reviews of the 
treatment teams in order to maintain fidelity to each treatment protocol. 

The 67 active duty military participants assigned to the in-hospital, cognitive treatment 
group received eight weeks of intense cognitive rehabilitation program. Their mean age 
was 24.7, 93% were male, 69% were White, 24% were African American, and 7% were 
“other” or not specified. Participants had either moderate to severe TBI or severe TBI 
with a Glasgow Coma score of 9.4. 

This in-hospital, cognitive treatment program consisted of a team of interdisciplinary 
practitioners, including a physiatrist, neuropsychologist, occupational therapist, speech 
pathologist, and rehabilitation assistants. The physical therapist and psychiatric and 
neurological experts were consulted when needed. The participants in the group were 
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expected to follow military standards and a structured daily routine. The structured 
daily routine consisted of physical fitness training, and group and individual therapies. 
Therapies included cognitive, psychotherapy, pragmatic speech, community re-entry, 
and integrated work programs. 

The 53 active duty military participants assigned to the in-home without VR 
intervention treatment group received a limited home rehabilitation program. Their 
mean age was 26, 96% were male, 70% were White, 17% were African American, 11% 
were Hispanic, and 2% were Native American. All had moderate to severe TBI or severe 
TBI with a mean Glasgow Coma score of 9.5. 

The in-home rehabilitation program included TBI education and individual counseling 
by a psychiatric nurse, who made weekly 30-minute phone calls to each participant. 
Educational materials and strategies to improve cognitive and organizational skills were 
given to participants. Participants were also trained in card game exercises and were 
encouraged to watch the news and read. They were also expected to resume daily 
physical fitness training. No participants in this group received vocational or work 
rehabilitation. 

The Salazar et al. study (2000) reported on the 12-month outcome evaluation. Sixty 
participants (90%) in the in-hospital, cognitive treatment group returned to work, 
whereas 50 of 53 participants (94%) in the in-home treatment group returned to work 
with an odds ratio of 0.514 (95% Cl = 0.126, 2.093). See Table 3 for the characteristics 
of the included study. 

Twamley et al. (2014, 2015) also conducted a RCT with parallel treatment groups. Fifty 
United States veterans with mild to moderate TBI from recent conflicts participated. 
Twenty-five were randomly assigned to the supported employment with CogSMART 
group (treatment group) and 25 were randomly assigned to the enhanced supported 
employment group. Only 42 participants completed the study, with 21 participants in 
each of the two groups. Time since most recent TBI was 4.08 years (SD=2.78) for the 
CogSMART group (treatment one group) and 5.04 years (SD=4.6 i) for the enhanced 
supported employment group (treatment two group) (p=0-377). The majority of 
participants in each group were men (96% in each group). The two groups were also 
similar in composition in terms of the number of participants who identified as 
Hispanic (36%). Twenty-four percent of the treatment one group and 48% of the 
treatment two group identified as White. The mean length of loss of consciousness 
across the last four traumatic brain injuries was six hours for the treatment one group 
and one hour for the treatment two group (p=o.274). There was a statistically 
significant difference between the two groups in terms of participants’ age. The 
treatment one group had a mean age of 29.68 (SD= 6.05), and the treatment two group 
had a mean age of 33.84 (SD=7. 89) (p = 0.042). A statistically significant difference was 
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also reported in education. The treatment one group had a mean of 12.88 years 
(£D= 1.27), versus 14.00 years (SD=i.8o) for the treatment two group (p=o.oi4). The 
odds ratio for this comparison was 0.881 (95% Cl = 0.235, 2.844). 

All participants in both groups received supported employment for one year. For the 
first three months of the intervention, the treatment one group received supported 
employment and CogSMART, while the treatment two group received additional 
supported employment sessions. CogSMART addressed post-concussive symptoms, 
prospective memory, attention and vigilance, learning and memory, and executive 
functioning (see cogsmart.com for more information). The treatment one group 
received one hour of CogSMART and one hour of supported employment per week for 
the first three months, and then one hour of supported employment for the rest of the 
year. The treatment two group received two hours of supported employment for the first 
three months, and then one hour of supported employment for the rest of the year. 
Separate employment specialists worked with each group. 

Each group lost four participants during the course of the study or at follow-up, which 
left each group with 21 participants each. There was no difference in competitive 
employment at 12 months, with both groups having 13 participants (52%) in 
employment. Thus, CogSMART did not improve the odds of employment for the 
treatment one group. The supported employment sessions that both groups received 
may have contributed to this finding. Further research with three groups— CogSMART 
only, CogSMART and supported employment, and supported employment only— needs 
to be conducted. 

Missing Data. All three studies had attrition. Man et al. (2013) lost five participants 
in each group during the study. Salazar et al. (2000) lost seven participants in the in- 
hospital cognitive intervention group and nine in the at-home intervention group. 
Twamley et al. (2014, 2015) lost four participants in each group. Participants who 
dropped out were not included in this analysis. Salazar et al. did not provide attrition 
information. Data on the Glasgow Coma Scale was incomplete for the Salazar et al. 
study, with scores for only 44 of the participants in the study available. The Twamley 
articles did not report Glasgow Coma Scale scores but instead provided length of total 
loss of consciousness injuries (U.S. military commonly report multiple traumatic brain 
injuries during deployment). 

Man et al. (2013) did not report on any demographic data such as gender, 
race/ethnicity, or education level. Twamley presented the demographic data previously 
mentioned, but the reporting across the two articles published on this study varied 
(Twamley et al., 2014, 2015). 
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For the Salazar et al. (2000) study, the primary outcome was employment status, and it 
was reported in binary form at twelve-month follow-up. None of the other articles 
concerning this study reported other time points (Braverman, 1999; Warden et al., 
2000). The secondary outcome data— mean hourly wage and total days employed— were 
not reported in this study. 

Man et al. (2013) used four categories to measure employment: (1) full-time 
employment, (2) full-time supported employment, (3) sheltered employment, and (4) 
unemployed or unable to work. For this review, full-time employment and full-time 
supported employment were summed to determine the number of competitively 
employed participants. Sheltered employment and unemployed/unable to be employed 
were summed for the unemployed group. Employment data were collected and reported 
at three time periods: one month, three months, and six months (See Table 1). 

Twamley et al. (2014, 2015) reported that employment status was collected at three, six, 
and twelve months. Twamley et al. reported three-month employment data in one study 
(2014). The second article states that three-month, six-month, and twelve-month 
outcome data would be presented, but Twamley et al. (2015) only reported twelve- 
month results in this article. Six-month data were not reported in either article, and 
further literature searches did not yield any documents reporting the six-month data. 
Only the binary data of whether participants were employed or not was collected. No 
other employment-oriented outcomes were measured, such as those of interest to this 
study— total days employed and mean hourly wage. 

studies reported employment status at different time periods ranging from one month 
to six months. Two studies (Man et al., 2013; Twamley et al., 2014) reported 
employment outcomes at three months. Salazar et al. (2000) and Twamley et al. (2015) 
reported employment outcomes at 12 months. Twamley et al. (2015) stated that three- 
month, six-month, and twelve-month outcome data would be reported, but only twelve- 
month employment findings were reported in this article. The overall analysis was 
performed on the latest time period for each study, which was 12 months for the Salazar 
et al. (2000) and Twamley et al. (2015) studies, and six months for the Man et al. study 
(2013). Sensitivity analysis was conducted to determine whether this was problematic. 
(See Table 1.) 

Lastly, none of the studies reported to be intent-to-treat studies. Two studies had 
attrition: Man et al. (2013) lost three participants in the artificial intelligence group and 
none of the participants in the psychoeducational group. Twamley et al. (2014, 2015) 
lost four in each of the two groups (CogSMART Plus group and enhanced supported 
employment group). Salazar et al. (2000) did not report attrition. This was an active 
military sample and the type of participants might have contributed to no attrition 
being reported. Data were used as reported. In the two studies where participants 


35 


The Campbell Collaboration | www.campbellcollaboration.org 



dropped out, the total number remaining in the sample was used. Thus, findings for the 
participants who remained are included in the Results section. 

4.1.3 Excluded Studies 

Sixty-four of the 67 study reports did not meet the inclusion criteria. The excluded 
studies included eight randomized controlled trials (RCT), 13 non-randomized pre-post 
two-group studies, one non-randomized pre-post three-group study, 22 pre-post one- 
group studies, 11 retrospective studies, five case studies, and one description of an 
intervention. Below, the RCT and non-randomized pre-post designs are summarized. 
The RCTs included cognitive rehabilitation (Cicerone et al., 2008; Vanderploeg et al., 
2008), multidisciplinary rehabilitation programs (Powell, Heslin, & Greenwood, 2002; 
Zhu, Poon, Chan, & Chan, 2007), early intervention (Wade, Crawford, Wenden, King, & 
Moss, 1997), motivational interviewing by way of the telephone (Bell et al., 2005), 
resource facilitation (Trexler, Trexler, Malec, Klyce, & Parrott, 2010), and individualized 
tailored treatment (Andersson, Emanuelson, Bjorklund, &Stalhammar, 2007). Five 
used control groups (Andersson et al., 2007; Bell et al., 2005; Powell et al., 2002; 
Trexler et al., 2010; Wade et al., 1997), and three used comparison treatment groups 
(Cicerone et al., 2008; Vanderploeg et al., 2008; Zhu et al., 2007). All but one study 
used civilians in their study (Andersson et al., 2007; Bell et al., 2005; Cicerone et al., 
2008; Powell et al., 2002; Wade et al., 1997; Trexler et al., 2010; Zhu et al., 2007). The 
other study used veterans (Vanderploeg et al., 2008). TBI severity ranged from mild to 
severe, with two studies not reporting severity. The sample size ranged from 68 to 483. 
These studies were conducted in China, Sweden, the United Kingdom, and the United 
States. Most RCTs were rejected because some of the participants were under 18 years 
of age (Andersson et al., 2007; Bell et al., 2005; Powell et al., 2002; Wade et al., 1997; 
Trexler et al., 2010; Zhu et al., 2007), or because some participants did not work at the 
time of injury (Andersson et al., 2007; Bell et al., 2005; Cicerone et al., 2008; Trexler et 
al., 2010; Wade et al., 1997; Zhu et al., 2007). One RCT was rejected because it 
combined RTW with school and did not present separate findings (Vanderploeg et al., 
2008). 

The 13 non-randomized pre-post two-group studies and the one non-randomized pre- 
post three-group study (High, Roebuck-Spencer, Sander, Struchen, & Sherer, 2006) will 
also be described. The interventions used in these studies included neuropsychologic 
interventions (Prigatano et al., 1984; Prigatano et al., 1994; Sarajuuri et al., 2005; 

Wilier, Button, & Rempel, 1999), community-based interventions (Heinemann, 
Corrigan, & Moore, 2004; High et al., 2006; O’Neill, Zugger, Fields, Fraser, & Pruce, 
2004; Ponsford, Harrington, Olver, & Roper, 2006), cognitive interventions (Cicerone, 
Mott, Azulay, & Friel, 2004; Dawson, Binns, Hunt, Lemsky, & Polatajko, 2013), 
occupational interventions (Radford et al., 2013), work entry interventions (Haffey & 
Abrams, 1991), comprehensive day interventions (Hashimoto, Okamoto, Watanabe, & 
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Ohashi, 2006), and an employment enhancing program with club houses (Niemeier et 
al., 2010). There was one non-randomized pre-post three-group study (High et al., 
2006), which will also be described. Ten studies used control groups (Cicerone et al., 
2004; Dawson et al., 2013; Hashimoto et al., 2006; Niemeier et al., 2010; O’Neill et al., 
2004; Prigatano et al., 1984; Prigatano et al., 1994; Ponsford et al., 2006; Radford et al., 
2013; Sarajuuri et al., 2005; Willler et al., 1999), and four compared treatments (Haffey 
& Abrams, 1991; Heinemann et al., 2004; High et al., 2006). All used civilians. TBI 
severity ranged from mild to severe. The sample size ranged from 13 to 319. These 
studies were conducted in Australia, Canada, Finland, Japan, the United Kingdom, and 
the United States. Most of these studies were rejected because not all participants were 
employed at the time of injury (Cicerone et al., 2004; Haffey & Abrams, 1991; 
Hashimoto et al., 2006; Heinemann et al., 2004; Ponsford et al., 2006; Prigatano et al., 
1984; Prigatano et al., 1994; Radford et al., 2013; Sarajuuri et al., 2005). Other reasons 
for exclusion included the following: not all participants were over the age of 18 years 
(Radford et al., 2013; Sarajuuri et al., 2005); employment category was unclear, which 
made it difficult to determine whether competitive employment was assessed or not 
(O’Neill et al., 2004); some participants were employed at the beginning of the 
treatment (Dawson et al., 2013); and separate findings were not provided for TBI 
(Niemeier et al., 2010; Wilier et al., 1999). 

Studies had to have a control or comparison group to be included in this systematic 
review. Thus, all of the pre-post only-group studies were excluded from the review. The 
interventions found among the 22 pre-post one-group studies included 
neurorehabilitation (Bjorkdahl, 2010; Ezrachi, Ben-Yishay, Kay, Diller, & Rattok, 1991; 
Geurtsen et al., 2011; Geurtsen et al., 2012; Klonoff, Lamb, & Henderson, 2000; 

Klonoff, Lamb, & Henderson, 2001; Perino, Zappala, Verne, & Rago, 2000); supported 
employment (Curl, Frasier, Cook, & Clemmons, 1996; Ellerd & Moore, 1992; Malec & 
Moessner, 2006; Wehman et al., 2003); club house (Vandiver & Christofero-Snider, 
2000); cognitive therapy (Braverman et al., 1999; Walker, Onus, Doyle, Clare, & 
McCarthy, 2005); transitional job coaching (Preston & Ulicny, 1992); in-home 
rehabilitation (Warden et al., 2000); comprehensive outpatient rehabilitation (Ben- 
Yishay, Silver, Piasetsky, & Rattok, 1987); multidisciplinary interventions (Christensen, 
1992); community-based interventions (Kolakowsky-Hayner, Wright, Shem, Medel, & 
Duong, 2012); interpersonal communication training; attention training; community 
training; and personal counseling (Rattok et al., 1992). The severity of TBI ranged from 
mild to severe, and sample sizes ranged from 9 to 603. Two studies were military 
(Braverman et al., 1999; Warden et al., 2000) and were one-group analyses of the 
Salazar et al. (2000) study included in this review. The other 20 studies were civilian. 
These studies were conducted in Australia, Denmark, Italy, the Netherlands, Sweden, 
the United Kingdom, and the United States. See Table 4 for more information 
concerning the characteristics of the excluded studies. 
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4-1.4 Location of Study 


Two of the three included studies were conducted in the United States (Salazar et al., 
2000; Twamley et al., 2014, 2015). The other study (Man et al., 2013) was conducted in 
China. 


4.1.5 Participants 

Sample sizes ranged from 40 to 120 participants, with a total sample size across the 
three included studies of 146. Two studies provided the proportion of males and mean 
age (Salazar et al., 2000; Twamley et al., 2014, 2015). The unweighted mean age for 
study participants was 28.56. The Salazar et al. and Twamley et al. samples were 
predominantly male, ranging from 93% to 96% male, with a mean of 95.25. Salazar and 
colleagues (2000) used active military samples from the United States, and Twamley et 
al. (2014, 2015) used U.S. military veterans. Man and colleagues (2013) used civilians in 
China. The Salazar et al. (2000) sample was predominantly African American and 
White, and there were more Hispanic/Latino and White participants in the Twamley et 
al. (2014, 2015) study. Man and colleagues (2013) did not report the ethnicity or race of 
their sample from China. The Twamley et al. (2014, 2015) and Man et al. (2013) studies 
reported severity of injury as mild to moderate. There were inconsistencies in the 
reporting of severity in the Salazar et al. (2000) study, with most places in the 
document reporting moderate to severe and one location in the Salazar et al. document 
reporting only severe injuries. 


4.2 RISK OF BIAS IN INCLUDED STUDIES 


A funnel plot was created, as stated in the protocol, to determine publication bias. (See 
Figures 3 and 5). too few articles were found to say that there was no publication bias 
because only three studies were included in the review. However, using log odds ratio 
on the horizontal axis and standard error on the vertical axis, the funnel plots were not 
symmetrical. Caution should be used in interpreting this funnel plot as an indication 
that there is no bias. The trim and fill analysis was performed and the point estimate for 
this analysis was 0.942 (df= 2) with 95% Cl = 0.420, 2.183; Q = 2.188, p = 0.335 and 
the funnel plot was asymmetrical, which indicates bias (See Figure 4.). 

Each of the included studies was assessed on the following five factors of risk of bias: 
allocation concealment, blinding, sequence generation, incomplete outcome data, and 
selective outcome reporting. Studies were rated as low risk, high risk, or 
unclear/unknown on each of the five factors. Based on the findings in Table 4, it is clear 
that there is risk of bias, especially in allocation of concealment, blinding, and selective 
outcome reporting. All three studies had a high risk of bias for allocation concealment 
(Man et al., 2013; Salazar et al., 2000; Twamley et al., 2014, 2015). All three were at low 
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risk for sequence generation because all three stated that they used randomized 
assignment. However, none of the studies discussed the process of randomization. All 
were at high risk of selective outcome reporting because Salazar et al. (2000) and 
Twamley et al. (2014, 2015) did not report the employment status of participants at all 
times the data were collected. Man et al. (2013) only collected and reported data during 
the first six months, not at 12 months. Further, none of the studies included additional 
employment information other than employment status. These studies did not report 
on hours worked, length of time worked, or wages paid, which would provide a better 
understanding of the success of the employment situations. 

Design, review status, publication type, and the presence of a control group were also 
examined. All were experimental studies. The Man et al. (2013) and Salazar et al. 

(2000) studies used a parallel design, in which another intervention was used. In the 
Man et al. study, the PVETS was used as the comparison (second intervention) group. 

In the Salazar et al. study, an in-home intervention was used. Both of these 
interventions were less intense than the treatment groups. The Twamley et al. study 
said it used a control group that received less intense treatment. However, it could be 
argued that framing the use of enhanced supported employment as a control group may 
have been inappropriate, since supported employment is an intervention. Thus, we 
considered all three studies to have alternate treatment (second intervention) groups 
that had the possibility of being as effective as the primary treatment group, which 
biased this systematic review. See Table 4 for a summary of all risk of bias factors. 


4.3 SYNTHESIS OF RESULTS 


The a priori plan was to conduct a meta-analysis on experimental and quasi- 
experimental studies with a treatment group and a control/comparison group. Three 
studies fit all criteria (Man et al., 2013; Twamley et al., 2014; Salazar, et al., 2000). 

Odds ratios, standard error, and a forest plot were computed since there was inadequate 
power to perform a meta-analysis. Comprehensive Meta-Analysis (Version 2) software 
was used to compute odds ratios and standard error and construct the forest plot. Below 
are the odds ratios for each study. 


Study 

Primary Intervention (1) and (2) Alternate Group 

Design 

Odds 

Ratio 

Standard 

Error 

P 

Man et al., 
2013 

(1) Artificial Intelligent Virtual Reality-Based Training 
Program and 

(2) Psychoeducational Vocational Training Program 

Experimental 

2.204 

0.800 

0.264 

Twamley 
et al.,2014 

(1) CogSMART and Supported Employment and (2) 
Enhanced Supported Employment 

Experimental 

0.817 

0.636 

0.353 

Salazar et 
al., 2000 

(1) Intensive In-Hospital Treatment and 

(2) At-Home Program 

Experimental 

0.514 

0.716 

0.749 
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None of the odds ratios were statistically significant. A comparison of the odds ratios 
was not conducted due to the small number of studies. No one intervention can be 
identified as being the most effective. Below, the odds ratios of each study are discussed. 

Man et al. (2013) hypothesized that participants in the virtual reality-based vocational 
training program would have better employment outcomes compared to participants in 
the conventional psychoeducational vocational training program. Four groups were 
used to organize employment status: unemployed, sheltered workshop, supported 
employment, and open (competitive) employment. No participants reported sheltered 
workshop involvement. The odds ratio above was computed with supported 
employment included in the competitive employment count, as stated in the methods 
section. The odds ratio indicated that participants in the virtual reality training were 
2.204 times as likely to be competitively employed than participants in the 
psychoeducational training program. Although the odds ratio indicated that virtual 
reality training was more effective than psychoeducational training, the rate of 
employment for the virtual reality program was only 30%, compared with 15% for the 
psychoeducational training. Thus, the average unemployment rate was 70% for the 
virtual reality training, and 85% for the psychoeducational training program. Even 
though the statistically non-significant odds ratio indicated that the virtual reality 
training was twice as effective as the psychoeducational training, the employment rate 
was still low. 

The Salazar et al. (2000) study hypothesized that of the two parallel groups, the 
intensive in-hospital rehabilitation program would have a greater number of 
participants returning to work than the at-home program. The statistically non- 
significant odds ratio indicated that participants in the in-hospital program were 0.514 
times as likely to be employed as participants in the at-home program. In addition, the 
employment rates of both intervention groups were high, with the majority of the in- 
hospital (89%) and at-home (94%) participants returning to work. 

Twamley et al. (2014) hypothesized that the supported employment plus CogSMART 
group would have better rates of job placement when compared to the enhanced 
supported employment group. The statistically non-significant odds ratio in this study 
indicated that participants in the supported employment plus CogSMART group were 
0.815 as likely to be competitively employed as participants in the enhanced supported 
employment group. Further, the supported employment plus CogSMART group had an 
average job placement of 55% across the two time periods (3 months and 12 months), 
while the enhanced supported employment group had an average job placement of 60%. 
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In summary, all odds ratios were statistically non-significant. No comparisons of the 
three included studies were performed. No one intervention was more effective, 
although all interventions led to competitive employment for some of their participants. 

4.3.2 Sensitivity Analysis 

This review did not have an adequate number of studies to conduct a meta-analysis to 
determine group differences. A larger number of studies might have been included in 
this systematic review if inclusion criteria had included (1) a broader range of brain 
injuries, and (2) a broader definition of employment to include sheltered employment 
and/or employability (i.e., volunteerism and engaging in post-secondary education). 
Broadening the diagnosis to include acquired brain injury, including non-traumatic 
diagnoses such as cerebral anoxia, aneurysm, carbon monoxide poisoning, cardiac 
arrest, and brain tumour (Bergquist et al., 2012; Geurtsen et al., 2012; Malec & 
Moessner, 2006; Rattok et al., 1992; Sloan et al., 2009), had the potential to increase 
the number of studies included. However, all five of the excluded studies that included 
participants with acquired brain injury did not meet other inclusion criteria. A few 
studies were one-group, no-comparison-group studies (Bergquist et al., 2012; Geurtsen 
et al., 2012), and others studies did not use competitive employment as the outcome 
(Geurtsen et al., 2012; Malec & Moessner, 2006; Rattok et al., 1992; Sloan et al., 2009). 
Broadening employment to include sheltered employment, volunteerism, and 
educational achievement would also have provided a larger number of studies to 
include. However, it would not have provided information on how effective 
interventions were at returning people with TBI to work in a competitive setting with 
people without TBI. The ultimate goal of vocational research interventions is to help 
people with disabilities such as TBI find employment with people without disabilities, 
rather than segregating people with TBI into sheltered employment. Furthermore, 
sheltered employment, volunteer work, and school do not have the same high standards 
as competitive employment. 


4.4 SUMMARY OF MAIN RESULTS 


Three RCTs comparing two intervention groups were found. The three RCTs consisted 
of an active military sample (Salazar et al., 2000), a veteran sample (Twamley et al., 
2014, 2015), and a civilian sample from China (Man et al., 2013). Interventions included 
an intensive in-hospital program versus an at-home program (Salazar et al., 2000); 
CogSMART plus supported employment versus enhanced supported employment 
(Twamley et al., 2014, 2015); and virtual reality-based training versus psycho- 
educational training (Man et al., 2013). Two studies involved U.S. military personnel: 
one used active military personnel (Salazar et al., 2000) and the other used a veteran 
population involved in recent conflicts (Twamley et al., 2014, 2015). The last study used 
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a civilian sample located in China (Man et al., 2013). All odds ratios were statistically 
non-significant. A meta-analysis was not performed due to the small number of 
included studies. Although all study interventions led to competitive employment, no 
one intervention was identified as more effective than the rest. A larger sample of 
studies is needed to provide a conclusive determination. None of the studies reported 
secondary employment outcomes: hours worked and wages earned. 

The Man et al. (2013) study trained individuals with TBI using two different methods to 
determine which intervention was more effective in assisting participants to gain 
employment. Most of the outcome variables measured were cognitive. One intervention 
consisted of a computer-based, 3-D, virtual-reality training program, and the other 
intervention was described as a conventional psycho-educational approach. The 
hypothesis was that individuals who were trained with the computer-based training 
program would have higher problem-solving skills and better employment outcomes 
than the group that received psycho-educational training. Employment outcomes were 
measured as competitive full-time employment, competitive part-time employment, 
supported employment, and sheltered employment. Competitive full-time and part- 
time employment and supported employment were considered competitive 
employment for the purposes of this review. 

Both groups were tested using a number of cognitive tests and an “on-site, work 
performance test.” The work-performance test included clerical tasks similar to those 
used for the training interventions. Testing was not conducted to evaluate whether the 
participants generalized problem-solving skills to a workplace or job. 

Neither group showed improvements in the on-site test, although there were differences 
in employment between the groups. This finding is important to consider, because 
teaching problem-solving skills that do not generalize to an actual work setting may be 
counter-productive and may expend valuable resources that should be used for other 
services, such as supported employment. Supported employment addresses a person’s 
difficulty with generalizing by providing training in skills that an individual with TBI 
needs on the job and not in simulated settings. 

Man and his colleagues (2013) reported that poor employment outcomes were related 
to participants’ lack of interest in obtaining employment, a poor economy, supported 
employment as a new initiative in Hong Kong, and legal limitations. If these assertions 
are correct, additional research is needed that explores participant interest in obtaining 
employment after China has fully accepted supported employment, and with 
participants who are not involved in legal situations that may inhibit them from seeking 
employment. In addition, future research should include outcomes such as wages, 
access to benefits, increased hours worked, job tenure, and job satisfaction. 
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Twamley and her colleagues’ description of supported employment and the manner in 
which they implemented the intervention are of concern (Twamley et al., 2014, 2015). 
Twamley reported that the CogSMART group received one hour of supported 
employment and one hour of the CogSMART intervention per week, for a total of two 
hours of intervention per week. The enhanced supported employment group received 
two hours of supported employment. Supported employment services for both groups 
was defined as one-hour sessions provided in a location of the individual’s choice, such 
as the career center, home, coffee shop, library, or VA clinic. 

Although supported employment can be provided away from the individual’s job site, as 
defined in the Workforce Innovation and Opportunity Act, no other information was 
provided on the types of supports that were delivered during the intervention. In other 
words, there is no information on what the employment specialists did with the 
participants during these sessions. Supported employment supports can include 
assessment, assistance with the job search/job development, job placement, on-the-the 
job training, and ongoing supports throughout the person’s job tenure. These supports 
are based on the needs of each individual and can be provided off as well as on the 
individual’s job site. In addition, the intensity of the intervention or amount of time 
spent providing services varies based on an individual’s needs and typically fades as the 
individual requires less assistance. Since no information was provided on the types of 
supported employment supports that the participants received, replication of the 
supported employment intervention is problematic. 

While the authors concluded that the CogSMART intervention was associated with 
reductions in post-concussive symptoms, there is no evidence that this intervention was 
associated with improved employment outcomes. Using supported employment as a 
component of the intervention in the comparative group is also problematic. It is 
uncertain whether the supported employment intervention or the CogSMART 
intervention affected the employment outcomes reported by the researchers. In 
addition, no statistical difference was found for the employment outcomes achieved by 
either group. Clearly, additional research is needed to determine if the CogSMART 
intervention affects the competitive employment outcomes of individuals with TBI. 

Salazar et al. (2000) evaluated the effectiveness of in-hospital cognitive rehabilitation 
versus an in-home rehabilitation program in returning veterans with TBI to 
employment. Patients were randomly assigned to the two interventions. The in-hospital 
group received physical fitness training, as well as group and individual cognitive, 
speech, occupation, and coping skills therapy. The afternoon sessions included 
integrated work therapy, including coordinated placement in various work settings 
similar to the individual’s military specialty. Participants in the in-home intervention 
were given recommended strategies to improve their cognitive skills, including training 
in card game exercises, reading books, and watching news programs. This group also 
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received weekly 30-minute telephone calls from a psychiatric nurse. The researchers 
hypothesized that the in-hospital group would experience greater RTW and fitness for 
military duty at a one-year follow-up. Results of the study found that there were no 
statistically significant differences in RTW for the two groups, with 90% of the hospital 
group and 94% of the in-home group returning to work at one-year post follow-up. 
Although this study reported high employment rates, the odds ratio of 0.514 was not 
significant (p = 0.353). 


4.5 OVERALL COMPLETENESS AND APPLICABILITY OL 
EVIDENCE 


4.5.1 Duration of Follow-Up 

All of the studies (Man et al., 2013; Salazar et al., 2000; Twamley et al., 2014, 2015) 
followed Participants for six months (Man et al., 2013) and 12 months (Salazar et al. 
2000; Twamley et al., 2014, 2015) after the end of the RTW intervention. Note that 
Twamley et al. (2014) stated that three-month, six-month, and twelve-month results 
were reported in the article, but only the twelve-month follow-up competitive 
employment status outcome data were reported in this article. In all of the studies, 
information is not provided on the latency between completion of the program and 
employment experiences during the follow-up period. It is unknown how long it took 
participants to obtain employment, or whether they were placed in a single or multiple 
employment positions within the 12 to 18 months. Future research should continue to 
follow participants for at least 12 months and should report the length of time taken to 
achieve employment, the number of placements, and the duration of each placement in 
order to assess the extent of engagement in the competitive workforce following 
treatment. 

4.5.2 Coverage of Outcomes 

This review focused on competitive employment as the primary outcome because it is 
the optimal level of real-world functioning and the most appropriate measure of the 
success of a RTW intervention. This diverges from much of the extant research, which 
tends to combine college student status, sheltered employment, and competitive 
employment. Some past studies have also used level of productivity as the outcome 
variable. Due to the number of ways in which RTW can be assessed, future research 
should report separate rates for all possible outcomes, rather than combining outcomes. 

Neither of the secondary outcomes— time to employment and hours worked— were 
analyzed due to lack of data. In order to understand the effectiveness of RTW vocational 
interventions, future research should also report time until employment and hours 
worked per week. 
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4-5-3 Origin 


Two of the three studies used military samples and were only applicable to military 
populations in the United States. Further, while the use of military samples is relevant 
today due to the Global War on Terror, it is unknown whether this intervention is 
effective with civilian samples. Although the Man et al. (2013) study used a Chinese 
sample, study samples need to be more diverse. Future research is needed with people 
from other countries, and with civilians. 


4.6 QUALITY OF THE EVIDENCE 


All the studies were RCTs with parallel treatment groups. However, there is some 
concern about the effectiveness of these interventions compared to a true control group. 
The three studies included in this systematic review used parallel groups that received 
an intervention. This is different from comparing a known inferior intervention or no 
intervention to an intervention. While this allows two effective interventions to be 
compared to determine which is best, it is not possible to determine the effect size of the 
interventions in the absence of a true control group. It would have been helpful to have 
control groups with no intervention as a point of comparison. 

Other issues that impact the quality of the evidence concern blinding, incomplete data, 
and selective outcome reporting. Blinding was only attempted by Salazar et al. (2000); 
Man et al. (2013) and Twamley et al. (2014, 2015) did not blind. Incomplete data was 
also an issue for the studies as a result of attrition, lack of demographic data, 
inconsistent use of the same severity measure, and data collection at different time 
points. Having this information would have increased the sample size and provided 
more information on which to compare studies using demographic data. Not having 
data at the same time point is problematic. More subjects could have gained 
employment from six to twelve months in the Man et al. (2013) study. Collecting and 
reporting employment status more frequently (e.g., at three months and six months) 
would help to understand the efficacy of the interventions. Salazar et al. (2000) and 
Twamley et al. (2014, 2015) did not provide this information. 

The studies were selective in their reporting of findings and data. All of the studies only 
reported on competitive employment status. No other employment data were reported. 
Employment status was the primary outcome, but this alone does not provide a clear 
understanding of RTW for persons with a TBI. number of hours worked and wages 
earned provide important information on effectiveness. An intervention that leads to 
employment in which a person is working 35 or more hours a week is important to note. 
Further, wage information is important to determine whether people with TBI will be 
able to earn enough money to live or lift themselves out of poverty. Thus, selective 
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reporting of employment status only, and not hours worked and wages earned, impedes 
assessments of efficacy. 

In sum, the strengths of these studies were that they were experimental, they randomly 
assigned participants to groups, they were peer reviewed, and they were published in 
academic research journals. The weaknesses of the studies include lack of blinding, 
incomplete data, selective reporting, and a lack of comparison groups. 


4.7 LIMITATIONS AND POTENTIAL BIASES IN THE REVIEW 
PROCESS 


4.7.1 Limitations 

The limitations of this systematic review are the study quality and number of included 
studies. Findings from this review may have been different if more studies had been 
included, and if the studies were of a higher quality than the ones included. 

4.7.2 Potential Biases in the Review Process 

Four issues in the review process might have contributed to potential bias: (1) including 
studies that had all been published in peer-reviewed journals, (2) coding studies into 
type of interventions, (3) determining whether studies were supply-side or demand-side 
interventions, and (4) language bias. 

An international grey literature search was conducted using internet databases that 
included or consisted only of grey literature, as well as an international search of peer- 
reviewed studies. The results of these searches were four studies published in peer- 
reviewed journals. Although studies were not limited to those published in English, a 
study that meets the criteria may not have been found because it was published in a 
language other than English or Spanish. 

Coding studies into types of interventions was discussed among the authors and 
colleagues. It was determined that supported employment intervention would be used 
as a broad category that included support from rehabilitation professionals on the 
supply-side and co-workers or employers on the demand-side. 

Language bias was evident in the review process. Only articles written in either English 
or Spanish were reviewed, and all databases were in English. 
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4.8 AGREEMENTS AND DISAGREEMENTS WITH OTHER 
STUDIES OR REVIEWS 


Although this systematic review had fewer studies than other TBI systematic reviews, 
this review had higher quality studies compared to previous systematic reviews (Brasure 
et al., 2013; Cancelliere et al., 2014; Fadyl & McPherson, 2009; Geurtsen et al., 2010; 
Kendall, Muencherberger, & Gee, 2006; Rohling, Faust, Beverly, & Damakis, 2009; Van 
Velzen, Van Bennekom, Edelaar, Sluiter, & Frings-Dresen, 2009; Virk, Williams, 
Brunsdon, Suh, and Morrow, 2015). The one exception was Virk and associates (2015), 
which examined cognitive interventions for people with acquired brain injury with 
attention as the outcome. Both this study and the Virk et al. (2015) study used only 
random control trials (RCTs). 

Productivity— which includes employment as well as other productivity activity, 
including education— was used as the outcome in the other RTW/productivity 
systematic reviews. Brasure et al. (2013) examined RCTs and cohort studies and used a 
more inclusive outcome variable— productivity. Kendall and colleagues (2006) also 
examined productivity and return to competitive employment in treatment and non- 
treatment studies. Fadyl and McPherson (2009) included RCTs, other quantitative 
studies, secondary analyses, and qualitative research, and examined a range of RTW 
outcomes including job retention and stability. The other systematic reviews examined 
intervention characteristics (Geurtsen et al., 2010), prognosis of return to work 
(Cancelliere et al., 2014), number of people with TBI who return to work (Van Velzen et 
al., 2009), and cognitive functioning (Rohling et al., 2009; Virk et al., 2015). Combining 
return to work and school attendance in productivity is problematic in these studies. 
Attending or completing school does mean a person with TBI will be able to obtain 
employment afterward. However, it would be helpful and more rigorous for TBI 
intervention studies to provide separate findings for those who obtain employment and 
those who are attending school. 

The Findings from the systematic reviews that examined vocational interventions for 
RTW indicated that there are interventions that are effective in returning people with 
TBI to competitive employment, but there was no one intervention that was more 
effective (Brasure et al., 2013; Fadyl & McPherson, 2009; Kendall et al., 2006). The 
present systematic review concurs with these past systematic reviews. It is worth noting 
that Brasure and colleagues (2013) mentioned that intensive cognitive rehabilitation 
showed promise as being an effective intervention for return to work. However, Virk et 
al. (2015) reported in their systematic review that none of the cognitive treatments were 
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currently effective at remediating any dimension of attention, which is an important 
piece of cognitive treatment. 

The use of competitive employment as the sole outcome was unique to the present 
systematic review. Kendall et al. (2006) included a variety of RTW variables in their 
systematic review, including competitive employment. Kendall et al. found that studies 
with more inclusive definitions of RTW (such as productivity) reported higher 
employment rates than studies in which competitive employment was the outcome. 
However, those who were competitively employed were more likely to continue to be 
engaged in employment than people who were engaged in other productive activities 
(Kendall et al., 2006). 

In sum, this systematic report offers tentative findings for military interventions and 
samples, but, as with other systematic reviews, offers no conclusive findings on the most 
effective intervention. This systematic review was different in that it included only RCTs 
and used a higher standard of employment outcome (i.e., competitive employment) 
than other studies. 
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5 Authors’ Conclusions 


5.1 IMPLICATIONS FOR PRACTICE AND POLICY 


Implications for Practice and Policy 

Job placement and vocational experts working in the field with clients who have TBI, 
especially moderate to severe TBI, are faced with a myriad of challenges. These include 
the obvious cognitive and social-emotional issues, but may also include self-awareness 
of TBI limitations, substance abuse, family and marital challenges, and service provider 
limitations in understanding how to place, train, and retain these individuals in 
employment. TBI is a lifelong disability, which requires (for the most part) ongoing 
support from the family and the service provider community, such as VR. 

This systematic review does not provide conclusive findings regarding an effective 
intervention. Although the odds ratios of the Salazar et al. (2000) and Twamley et al. 
(2014, 2015) studies were below one— which means that the alternate interventions 
were more effective— the treatment and intervention employment rates were higher 
than the Man et al. (2013) study, with the employment rate ranging from 55% for the 
CogSMART plus supported employment intervention (Twamley et al.) to 94% for the at- 
home intervention in the Salazar et al. study. Hence, we conclude that all employment 
interventions lead to employment, though we cannot identity the most effective 
treatment. Each study used a different intervention and population. 

The summary of excluded studies indicates that not all interventions currently in use 
were represented in this study. In addition, these three studies provide limited 
information to assist service providers provide ongoing supports (e.g., information 
about the intensity, frequency, or nature of these supports in the long term). Additional 
work is needed with standard follow-up times (such as three, six, 12, and 18 months) to 
effectively compare findings. 

These three studies have limited implications for practice and policy. Although they 
were randomized control studies, the populations were substantially limited to military 
subjects, who present with significantly different comorbidities and unique challenges 
(such as posttraumatic stress disorder), which can often skew whether the population is 
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actually brain injured. The previous competitive employment histories of many of the 
subjects were also unclear. A meta-analysis of all of the subject outcomes might be 
helpful in this regard. 


5.2 IMPLICATIONS FOR RESEARCH 


Some of the interventions used in this review— in-home (Salazar et al., 2000), in- 
hospital (Salazar et al., 2000), and supported employment (Twamley et al., 2014, 2015) 
interventions— are innovative and promising approaches to assist individuals who 
return to work. However, it remains difficult to find replication efforts or evidence that 
these approaches are being widely used by individuals with TBI. To improve access to 
this option, there is a need to establish mechanisms for educating and training 
individuals with TBI and their family members, as well as VR professionals and direct 
service staff. Individuals with TBI and family members need information about effective 
interventions (e.g., upon discharge from an outpatient or inpatient medical unit, and at 
follow-up at selected intervals, such as three months, six months, one year, and five 
years post-discharge). This may offer an opportunity to assist some individuals with 
returning to their preinjury employer in the same or a new position. Based on our 
previous research concerning VR professionals’ beliefs, knowledge, and willingness to 
engage in evidence-based practices (Graham, Inge, Wehman, Murphy, Revell, & West, 
2013; Inge, Graham, Erickson, Sima, West, Cimera, In Press), VR professionals, first 
and foremost, must understand and embrace the basic principles associated with 
effective interventions (e.g., power of workplace supports). They must become skilled at 
successfully individualizing services and supports that focus on the career goals and 
preferences of individuals. 

There is a need for more evidence-based prospective studies that compare the efficacy 
of different interventions based on the work outcomes of persons with TBI. Studies that 
compared the outcomes of supported employment versus competitive employment that 
occurs without job coach supports would be helpful, as would studies comparing self- 
employment outcomes versus competitive employment outcomes. There is also a need 
for studies that examine whether the time of intervention makes a difference to the 
outcome. In addition, the use of peer mentors, the role of co-workers, and the role of the 
family-friend network all need to be investigated longitudinally. Further long-term 
follow-up, beyond the twelve months after intervention, is also needed to provide 
information on the use of follow-up employment services and employment outcomes. 

There is a clear need for public funding sources and services for this population beyond 
the federal/ state VR program. The lack of funding for work services is a major problem, 
given that many third-party insurance companies do not cover these services. From a 
policy perspective, this is an enormous obstacle that will need to be resolved. 
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More creative ways to measure employment services are also needed, besides the use of 
employment rate. Employment rates often skew our understanding of the person with 
TBI’s learning trajectory. For example, a monthly employment ratio— total number of 
months the person could be employed versus actual months of employment— would 
allow for a more sensitive assessment. 

There needs to be significantly more effort aimed at training rehabilitation counselors, 
special education teachers, transition specialists, case managers, and psychologists on 
the work capacity and potential of persons with TBI. The initial work performance of 
many individuals with TBI is likely to be a failure but eventually can grow into success, 
although not without knowledge of workplace supports. These supports involve 
understanding the government’s role, the agency or clinical role, and the individual’s 
role and perspectives. 

There are several recommendations for the direction of research concerning RTW for 
adults with TBI. First, VR researchers must improve the quality of research by 
conducting randomized controlled trials (RCTs) on RTW interventions with adults with 
TBI. Second, separating competitive employment from school attendance would 
provide a more accurate estimate of RTW. Third, military interventions should be 
conducted with civilian samples in order to determine their effectiveness in the civilian 
population. Fourth, research is needed from other populations outside the United 
States. Fifth, studies need to include employment stability as an outcome. This can be 
measured by the amount of hours and days of post-employment services required by 
each participant, as well as the length of time in a job. Last, studies need to collect and 
report data at standard intervals (e.g., six months, 12 months, 18 months, etc.). This 
would allow reviews to provide a more accurate account of the effectiveness of the 
different interventions. Last, future RTW VR studies should report: 

• Time to employment 

• Hours worked 

• Separate rates of competitive employment, sheltered employment, 
educational training, etc. 

• Continued follow-ups at 12 months 

• Employment outcome and employment characteristics, collected every six 
months for at least 18 months 

• The number of employment experiences during the follow-up period, 
including the duration of each experience 

• Employment stability, measured by length of time worked at each job and 
number of jobs since intervention at each follow-up. 
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8 Tables 


8.1 TABLE l: TIME PERIODS OF EMPLOYMENT OUTCOMES 
REPORTED BY STUDY 


Study 

Time 1 

Time 2 

Time 3 

Man et al., 2013 

1 month 

3 months 

6 months 

Salazar et al., 2000 

12 months 



Twamley et al., 2014, 2015 

3 months 


12 months 
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8.2 TABLE 2: SEARCH STRINGS USED FOR FIRST SEARCH 


*Multiple terms in a cell indicate that each term was used methodically in combination 


with the other related terms 


First Term 

Second Term* 

Third Term* 

employee disability management 
practice 

TBI 

employment, occupation, return to 
work, supported employment, job 
coaching, job accommodation, 
multiple disciplines of rehabilitation, 
vocation 


TBI 

employment, occupation, return to 
work, vocation 

advocacy services, case coordination systems, 
MVCCS, cognitive therapy, Jacob’s club house, 
physical therapy, occupational therapy, rehabilitation, 
counseling, speech language therapy, support 
services, therapeutic activities, intervention, adaptive 
technology, adaptive devices, cognitive rehabilitation 

brain damage 

employment, occupation, return to 
work, supported employment, job 
coaching, job accommodation, 
multiple disciplines of rehabilitation 


brain damage 

employment, return to work 

advocacy services, case coordination systems, 
MVCCS, cognitive therapy, Jacob’s club house, 
physical therapy, occupational therapy, rehabilitation, 
counseling, speech language therapy, support 
services, therapeutic activities, intervention, adaptive 
technology, adaptive devices, cognitive rehabilitation 

brain damage 

occupation 

intervention, adaptive technology 

acquired brain injury 

employment, occupation, return to 
work, rehabilitation 


acquired brain injury 

employment, occupation 

intervention 

head injury 

employment, return to work 


head injury 

employment 

intervention 

head injury 

return to work 

rehabilitation 

head injury 

return to work 

cognitive rehabilitation 

head injury 

return to work 

physical therapy 
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8-3 


TABLE 3 : CHARACTERISTICS OF INCLUDED STUDIES 


Man et ah, 2013 


Methods 

Experimental - Randomized control trial with parallel treatment groups 

Participants 

Hong Kong sample: N - 50, age range: 18-55 


Treatment 1 group: n - 25. No group characteristics were reported. 


Treatment 2 group: n - 25. No group characteristics were reported. 

Interventions 

Treatment 1: Artificial intelligent virtual reality-based vocational training 
system (AIVST) consisted of twelve 20 to 25 minute sessions. AIVST program 
determined where each subject should begin vocational training, focused on 
providing self-efficacy by mastering skills and progressing to higher skill levels. 


Treatment 2: The intervention is a 12-session program that is structured and has a 
“conventional psycho-educational vocational training programme” focusing on 
training, practice and review of specific vocational skills (p. 1017). 

Outcomes 

Primary: problem-solving skills 


Secondary: employment status 

Notes 

Five subjects from each group dropped out before the end of study. This left 20 in 
each group. 


Salazar et al., 2000 


Methods 

Experimental - Randomized control trial with parallel intervention groups 

Participants 

Military sample: A/ = 120 


Treatment 1 group: n = 67; mean age = 24.7; 93% were male; 69% White, 

24% African American, 7% not specified 


Treatment 2 group: n = 53; mean age = 26; 96% were male; 70% White, 17% 
African American, 11% Hispanic, 2% Native American. 

Interventions 

Treatment 1: Residential in-hospital intervention with neuro-cognitive 
component and vocational rehabilitation 


Treatment 2: Home rehabilitation that consisted of TBI education material and 
counseling from a psychiatric nurse with no vocational rehabilitation 

Outcomes 

Primary: Employment status and fitness for duty 


Secondary: mean length of time employed; mean hours worked 

Notes 

Salazar et al. 2000 is not consistent in the article in how it labels the TBI 
severity. One time it states that all have severe TBI and in two other places it 
states that the injuries are moderate to severe. Another article on this RCT 
study states that the injuries are moderate to severe (Warden, Salazar, Martin, 
Schwab, Coyle, Walter, and the DVHIP Study Group, 2000). 
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Twamley et al., 2014 


Methods 

Experimental - randomized control trial 

Participants 

Military veteran sample: N = 50 


Treatment 1 group: n = 25; mean age = 29.4; 93.8% were male; 56.3% 
Caucasian, 37.5% Hispanic, 6.2% Not specified. 


Treatment 2 group: n = 25; mean age = 34.3; 94.4% were male; 61.1% 
Caucasian, 33.3% Hispanic, % Not Specified. 

Interventions 

Treatment 1: 12 weeks of Cognitive Symptom Management and Rehabilitation 
Therapy (CogSMART) and 12 months of 1 hour per session of supported 
employment. 

T reatment 2: 3 months of enhanced supported employment by employment 
specialist. 

Outcomes 

Primary: post-concussive symptoms 


Secondary: cognitive and functional capacity, and job placement 

Notes 

There are discrepancies in the reporting of race and ethnicity between the two 
published articles concerning this RCT (Twamley, Jak, Delis, Bondi, & Lohr, 
2014; Twamley et al., 2015). Demographic data including race and ethnicity 
were used from Twamley et al. 2014, since it had job placement as an 
outcome. 
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8.4 TABLE 4: CHARACTERISTICS OF EXCLUDED STUDIES 


Study 

Country 

Reason for Exclusion 

Total 

Carlson & Buckwald, 

1993 

United States 

Case Study 


Fraser & Baarslag- 
Benson, 1994 

United States 

Case Study 


Kowalske, Plenger, 

Lusby, & Hayden, 2000 

United States 

Case Study 


Kreutzer, Wehman, 
Morton, & Stonnington, 
1988 

United States 

Case Study 


Subtotal: 4 

Preston & Ulicny, 1992 

United States 

Data collection occurred before 48 
days after program ended 


Subtotal: 1 

Chao, 2011 

United States 

Did not measure employment/ 
unemployment status 


Wilier, Button, & Rempel, 

United States 

Did not measure employment/ 


1999 

unemployment status 


Subtotal: 2 

Bergquist, Micklewright, 
Yutsis, Smigielski, Gehl, 

& Brown, 2012 

United States 

Did not separate TBI 


Geurtsen, van Heugten, 
Martina, Rietveld, Meyer, 

& Geurts, 2012 

The Netherlands 

Did not separate TBI 


Malec& Moessner, 2006 

United States 

Did not separate TBI 


Rattok, Ross, Ezrachi, 
Silver, Lakin, Vakil, 
Piasetsky, & Diller, 1992 

United States 

Did not separate TBI 


Sloan, Callaway, Winkler, 
McKinley, Ziino, & Anson, 
2009 

Australia 

Did not separate TBI 


Trexler, Trexler, Malec, 
Klyce, & Parrott, 2010 

United States 

Did not separate TBI 


Subtotal: 6 

Cicerone, Mott, Azulay, & 
Friel, 2004 

United States 

Employment/student combined 
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Study 

Country 

Reason for Exclusion 

Total 

Hashimoto, Okamoto, 
Watanabe, & Ohashi, 

2006 

Japan 

Employment/student combined 


Heinemann, Corrigan, & 
Moore, 2004 

United States 

Employment/student combined 


Prigatano, Fordyce, 

Zeiner, Roueche, 

Pepping, & Wood, 1984 

United States 

Employment/student combined 


Sherer, Evans, Leverenz, 
Stouter, Irby Jr., Lee, & 
Yablon, 2007 

United States 

Employment/student combined 


Vanderploeg, Schwab, 
Walker, Fraser, Sigford, 
Date, ... Warden, 2008 

United States 

Employment/student combined 


Subtotal: 6 

O’Neill, Zuger, Fields, 
Fraser, & Pruce, 2004 

United States 

Mixed employment 


Vandiver & Christofero- 
Snider, 2000 

United States 

Mixed employment 


Subtotal: 2 

Abrams, Barker, Haffey, & 
Nelson, 1993 

United States 

Not 100% employed pre-injury 


Ben-Yishay, Silver, 
Piasetsky, & Rattok, 1987 

United States 

Not 100% employed pre-injury 


Cicerone, Mott, Azulay, 
Sharlow-Galella, Ellmo, 
Paradise, & Friel, 2008 

United States 

Not 100% employed pre-injury 


Dawson, Binns, Hunt, 
Lemsky, & Polatajko, 

2013 

Canada 

Not 100% employed pre-injury 


Niemeier, DeGrace, 

Farrar, Ketchum, Berman, 
& Young, 2010 

United States 

Not 100% employed pre-injury 


Geurtsen, van Heugten, 
Martina, Rietveld, Meijer, 

& Geurts, 2011 

The Netherlands 

Not 100% employed pre-injury 


Haffey & Abrams, 1991 

United States 
researchers - data 
from United Kingdom 
& United States 

Not 100% employed pre-injury 
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Study 

Country 

Reason for Exclusion 

Total 

High, Roebuck-Spencer, 
Sander, Struchen, & 
Sherer, 2006 

United States 

Not 100% employed pre-injury 


Klonoff, Lamb, & 
Henderson, 2001 

United States 

Not 100% employed pre-injury 


Micklewright, Yutsis, 
Smigielski, Brown, & 
Bergquist, 2011 

United States 

Not 100% employed pre-injury 


Phipps & Richardson, 

2007 

United States 
researchers - study 
took place in Canada 

Not 100% employed pre-injury 


Ponsford, Harrington, 
Olver, & Roper, 2006 

Australia 

Not 100% employed pre-injury 


Prigatano, Klonoff, 

O’Brien, Altman, Amin, 
Chiapello, ... Mora, 1994 

United States 

Not 100% employed pre-injury 


Sarajuuri, Kaipio, 

Koskinen, Niemela, 

Servo, & Vilkki, 2005 

Finland 

Not 100% employed pre-injury 


Walker, Onus, Doyle, Clare, Australia 
& McCarthy, 2005 

Not 100% employed preinjury 


Wall, Niemczura, & 
Rosenthal, 1998 

United States 

Not 100% employed pre-injury 


Wehman, Kreutzer, West, 
Sherron,... Killam, 1989 

United States 

Not 100% employed pre-injury 


Wehman, Kregel, Keyser- 
Marcus, Sherron-Targett, 
Campbell, West, & Cifu, 
2003 

United States 

Not 100% employed pre-injury 


Subtotal: 

18 

West & Anderson, 2005 

United States 

No data collection 


Subtotal: 1 

Andersson, Emanuelson, 
Bjorklund, & Stalhammar, 
2007 

Sweden 

Outside of age range 


Bell, Temkin, Esselman, 
Doctor, Bombardier, 
Fraser, Hoffman, Powell, 

& Dikmen, 2005 

United States 

Outside of age range 
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Study 

Country 

Reason for Exclusion 

Total 

Bjorkdahl, 2010 

Sweden 

Outside of age range 


Christensen, 1992 

Denmark 

Outside of age range 


Ezrachi, Ben-Yishay, Kay, 
Diller, & Rattok, 1991 

United States 

Outside of age range 


Gamble & Moore, 2003 

United States 

Outside of age range 


Klonoff, Lamb, & 
Henderson, 2000 

United States 

Outside of age range 


Kolakowsky-Hayner, 
Wright, Shem, Medel, & 
Duong, 2012 

United States 

Outside of age range 


Murphy, Chamberlain, 
Weir, Berry, Nathaniel- 
James, & Agnew, 2006 

United Kingdom 

Outside of age range 


Najenson, Groswasser, 
Mendelson, & Hackett, 
1980 

Israel 

Outside of age range 


Perino, Zappala, Verne, & 
Rago, 2000 

Italy 

Outside of age range 


Powell, Heslin, & 
Greenwood, 2001 

United Kingdom 

Outside of age range 


Radford, Phillips, 
Drummond, Sach, 

Walker, Tyerman, 

Haboubi, Jones, 2013 

United Kingdom 

Outside of age range 


Reid-Arndt, Schopp, 
Brenneke, Johnstone, & 
Poole, 2007 

United States 

Outside of age range 


Wade, King, Wenden, 
Crawford, & 

Caldwell, 1998 

United Kingdom 

Outside of age range 


Zhu, Poon, Chetwyn, 

Chan, & 

Chan, 2007 

China 

Outside of age range 


Subtotal: 

16 

Landa-Gonzalez, 2001 

United States 

Single subject 


Subtotal: 1 

Warden, Salazar, Martin, 
Schwab, Coyle, & Walter, 
2000 

United States 

Non-vocational rehabilitation 
intervention; in-home treatment group 
from a included study (Salazar et al., 
2000) 
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Study 

Country 

Reason for Exclusion 

Total 

Subtotal: 1 

Braverman, Spector, 


Same data set as included study 


Warden, Wilson, Ellis, 
Bamdad, & Salazar, 1999 

United States 

(Salazar et al., 2000) 


Subtotal: 1 

Altman, Swick, Parrot, 
Malec, 2010 

United States 

Retrospective study 


Watanabe, 2013 

Japan 

Retrospective study 


Wehman, Kregel, 

Sherron, Nguyen, 

Kreutzer, Fry, & Zasler, 
1993 

United States 

Retrospective study 


Subtotal: 3 

Curl, Fraser, Cook, & 
Clemmons, 1996 

United States 

Pre-post one-group-only design 


Ellerd & Moore, 1992 

United States 

Pre-post one-group-only design 



Subtotal: 2 


Total: 64 
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8.5 TABLE 5: RISK OF BIAS OF INCLUDED STUDIES 


Study 

Allocation 

Concealment 

Blinding Sequence 
Generation 

Incomplete 

Data 

Selective 

Outcome 

Reporting 

Design 

Review 

Status 

Type of 
Publication 

Type of Group 

Man et al., 
2013 

Unclear 

High 

Risk 

Low Risk 

Low Risk 

High Risk 

RCT with 

parallel 

groups 

Peer 

Reviewed 

Journal Article 

Comparison Group 
(second intervention) 

Salazar et 
al., 

2000 

Unclear 

High 

Risk 

Low Risk 

Low Risk 

High Risk 

RCT with 

parallel 

groups 

Peer 

Reviewed 

Journal Article 

Comparison Group 
(second intervention) 

Twamley 
et al., 

Unclear 

High 

Risk 

Low Risk 

Low Risk 

High Risk 

RCT with 
parallel 

Peer 

Reviewed 

Journal Article 

Comparison Group 
(second intervention) 


2014, groups 

2015 


84 


The Campbell Collaboration | www.campbellcollaboration.org 


8.6 TABLE 6: GROUP RESULTS 


Intervention 

Odds 

Ratio 

Log Odds 
Ratio 

Standard 

Error 

P 

95% Cl 

Variance 

Population 

Design 

Man et al., 2013 

2.204 

0.790 

0.800 

0.264 

0.512, 

11.51 

0.640 

Civilian 

ROT 

Salazar et al., 
2000 

0.514 

-0.665 

0.716 

0.353 

.126, 

.093 

0.513 

Military 

ROT 

Twamley et al., 
2015 

0.817 

-0.202 

0.636 

0.749 

.232, 

.860 

0.405 

Military 

ROT 
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9 


Data and Analyses 


Table y: Forest Plot 








Odds 

ratio 

Lower 

limit 

Upper 

limit 

Z- Value 

p-Value 

Man 

2.429 

0.512 

11.511 

1.118 

0.264 

Salazar 

0.514 

0.126 

2.093 

- 0.929 

0-353 

Twamley 

0.815 

0.232 

2.860 

-0.320 

0-749 


0-937 

0.420 

2.090 

-0.159 

0.874 


O dds ratio and 95% TT 
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Appendix 3. Coding Form for Primary Research Characteristics for Employment 
Interventions for Return to Work in Working-Aged Adults Following Traumatic Brain 
Injury (TBI) Systematic Review 

Author (s): 

Title: 

Date of Pub: 1 
Source: 1. Journal 

2. Book or Book Chapter 

3. Master Thesis/Doctoral Dissertation 

4. Presentation 

5. Government Agency Report 

6. Organization Report 

Intervention: 

Employment Intervention: 1. Yes 2. No (will be excluded from study) 

Participants Had TBI: 1. Yes 2. No (will be excluded from study) 

Comments: 

Type of Intervention: _Supportive Employment 

Provider: Demand Side Supply Side 

Duration of Intervention: 

Design: 

Randomized Controlled Trials (RCTs) 

Quasi-Experimental 

Single Subject Experimental 

Other (will be excluded from study) 

Sampling: 

Random, specify type 

Other, explain 

Allocation Concealment Yes (Low Risk) No (High Risk) Unclear/Unknown 

Low Risk Descriptions: 

Central allocation (telephone, web-based) 

Sequentially numbered, opaque and sealed envelopes 
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High Risk Descriptions: 

Using an open random allocation schedule (list of random numbers) 

Assignment envelope were not opaque and/or sealed 

Alternation or Rotation 

Date of Birth 

Case Record Number 

Random Numbers Table 

Other explicitly unconcealed assignment procedure, specify 

Unclear/Unknown Description 

Inadequate information to make a judgement. 

Blinding: Yes (Low Risk) No (High Risk) Unclear/Unknown 

Low Risk Descriptions: 

No blinding, but reviewers determine there is little chance that that outcome and 

outcome measurements are influenced by lack of blinding. 

Participants and key study personnel blinding occurred and little chance that blinding 

was broken. 

Some blinding occurred, but either participants or key personnel were not blinded and 

non-blinding had little chance of introducing bias. 

High Risk Descriptions: 

There is a lack of or incomplete blinding and there is a chance that outcome or outcome 

measurement will be influenced. 

There was an attempt to blind participant and key researchers but there is a good 

chance that blinding was broken. 

There was partial blinding and there is a good chance that this introduced bias. 

Unclear/Unknown Description 

Inadequate information to make a judgement. 

Study did not address outcome of interest. 

If blinded, check those who were blinded: 

Researcher 

Participant 

Intervener 

Assessor 

Employer 
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Other (explain) 

Sequence Generation: 

Yes (Low Risk) No (High Risk) Unclear/Unknown 

Low Risk Descriptions: 

Random Numbers Table 

Random Number Generated 

Coin Flip 

Other, specify 

High Risk Descriptions: 

Sequence generated by odd/ even birthdate 

Sequence generated by date of enrolment 

Sequence generated by CRN or other rule based method. Specify 

Assignment by participant preference 

Assignment by clinical judgement 

Assignment by pre-assessment tool 

Assignment by availability of the intervention 

Unclear/Unknown Description 

Inadequate information to make a judgement. 

Recruitment Pool: 

Referral Criterion Waiting List 

Existing Group Volunteer Not Reported 

Other, Explain 

Comments: 

Number of Intervention Sites: 

If random sampling use, was the same sampling procedure used at all sites? Yes 

Source of Sample 

1. Public Agency 

2. Private Agency 

3. Not Reported 

4. Other (explain) 

Type of Comparison Group Yes No 

If yes, check appropriate type 
Treatment as usual 
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No intervention 

Alternate Service, Explain 

Incomplete Outcome Data: 

Yes (Low Risk) No (High Risk) Unclear/ Unknown 

Low Risk Descriptions 

No missing outcome data 

Equal amount of missing data across groups and all groups have similar justifications 

for missing data. 

Dichotomous outcome data - ratio of proportion missing to observed is not relevant 

enough to bias the effect estimate. 

Continuous outcome data - not relevant enough to bias effect size estimate. 

Missing data have been appropriately imputed. 

High Risk Descriptions 

Unequal amount of missing data across groups and there is a good chance that this 

missing data is related to outcome across groups. 

Dichotomous data - the ratio of the proportion missing to observed is substantial 

enough to introduce bias. 

Continuous data - the amount of missing outcome data is large enough to introduce 

bias. 

Conducting an “As-treated” analysis with substantial discrepancy from random 

assignment. 

Inappropriate use of simple imputation. 

Unclear/Unknown Descriptions 
Information not reported. 

Incomplete reporting of attrition and exclusions such as no justification of missing 

outcome data. 
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Table l: Group Characteristics: 



Group 

Type 

Mean 

Age 

Age 

Range 

Males 

96 (n) 

Pre- 

test 

n 

Post- 

test 

n 

Attrition 

n 

Follow 

-Up 

n 

F-UP 

Attrition 

n 

Group 










Group 










Group 










Group 











SUBJECT CHARACTERISTICS 
SES 


Gender 

Group l 

Group 2 

Female 

% 

% 

Male 

% 

% 

Race/Ethnicity 

Group l 

Group 2 


% 

% 


% 

% 


% 

% 


% 

% 


% 

% 


% 

% 


% 

% 


Comments: 


Education 

Some High School Comments: 

High School Graduate/GED 
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Some College 

Graduated College 

Location of Treatment 

Urban Rural 

Suburban Other (explain) 

Comments: 

Where Injury Occurred: 

Motor Vehicle/Traffic 

Fall 

Combat/T error 

Assault 

Colliding with moving or stationary object 

Severity of Injury 

Mild Comments: 

Mild to Moderate 

Moderate 

Not Reported 

Other (explain) 

Subject Population: 

Civilian 

Military /Veteran 

Pre-Injury Employment 

Intervention Characteristics: 

Page 

Average length of intervention program 

Length of time of participation activity 

Number of Sessions 

Intervention Activities included 


Type of Professional conducting intervention 

Sample Status Assessed 

Only those that obtained full-time employed 
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Mixed - those that obtained full-time or part-time employed 

Primary Type of Employment 
Retail Trade 

Transportation & Warehousing 

Information 

Finance 

Professional 

Education & health 

Leisure & Hospitality 

Construction 

Manufacturing 

Local government 

State government 

Federal government 

Other Service: 

Not Reported 

Comments: 

XIII. Employment Outcome Measure(s): 

Employment outcome: 

1. Employed - dichotomous - 

a. Number employed 

b. Number unemployed, still 

2. Length of Time to Place in Employment: 

3. Length of Time Employed: 

4. Hours worked per week: 

5. Dropped Out Before Placement Occurred: 

Comments: 

XV. Selective Outcome Reporting 
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Yes (Low Risk) No (High Risk) Unclear/Unknown 


Low Risk Description 

Study protocol is available and all pre-specified outcomes that are of 

interest are reported in the pre-specified way. 

Study protocol is not available but all expected outcomes along with pre- 
specified outcomes are reported clearly. 

High Risk Description 

Reporting of pre-specified outcomes is not complete. 

Not all of pre-specified outcomes have been reported. Those that have 

been reported used a subset of data, used measured not pre-specified, or 
analysis that was not pre-specified. 

A t least one primary outcome was not pre-specified. 

A t least one outcome was not completely reported so that information can 
not be entered into meta-analysis. 

Study does not report findings for a key outcome that one would expect to 

have been reported for that particular type of study. 

Unclear/Unknown Description 

Inadequate information to make a judgement. 


Comments: 


XVI. Fidelity of Implementation 
Intervention implemented as described 

No 

Yes 

Not Reported 

Comments: 


96 


The Campbell Collaboration | www.campbellcollaboration.org 



Appendix 4: Coding Form for Primary Research Characteristics for Employment Interventions 
for Return to Work in Working-Aged Adults Following Traumatic Brain Injury (TBI) Systematic 
Review 


BASIC INFORMATION 
Author (s): 

Man, D. W. K., Poon, W. S., & Lam, C. (2013). The effectiveness of artificial intelligent 3-D 
virtual reality vocational problem-solving training in enhancing employment opportunities for 
people with traumatic brain injury. Brain Injury , 27, 1016-1025. DOI: 
10.3109/02699052.2013.794969 

Date of Pub: 2013 


Source: i._ 

x Journal 

2 . . 

Book or Book Chapter 

3-. 

Master Thesis /Doctoral Dissertation 

4-. 

Presentation 

5-. 

Government Agencv Report 

6.. 

Organization Report 


Intervention: 

Employment Intervention: 1. x Yes 2. No (will be excluded from study) 

Intervention (brief description): Participants randomly assigned to (1) AIVST or (2) 
PEVST group. Baseline information obtained. AIVST was the artificial intelligent virtual reality- 
based vocational training system. PEVTS was “a structured and content-similar conventional 
psycho-educational vocational training programme” (p. 1017). There were 12 session models. 
The training for both groups was similar, with included training in specific vocational skills, 
practice in those skills, and review of learned skills. AIVST group received twelve 20 to 25 
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minute training sessions and PEVTS received “conventional psycho-educational programme” (p. 
1017). 

AIVST captured variables, determines where each participant should begin training, provides 
feedback, and focuses on improving self-efficacy through achievement of skills and progressing 
to a higher skill level. PEVTS program used a training manual by a vocational trainer. 
Participants received tutorials with specific instructions, used structured games on problem 
solving, and practiced skills and routines. 

Intervention Type: 1. x Supply Side 2. Demand Side 

Duration of Intervention: 12 sessions 

Length of time of participation activity: 20 to 25 minutes 

Design: 

Randomized Controlled Trials (RCTs) x Randomized Controlled Trial (RCT) with Parallel 

Treatment Group 

Quasi-Experimental 

Other (will be excluded from study) 

Sampling: 

^Random, specify type Simple 

Other, explain 

Allocation Concealment Yes (Low Risk) x No (High Risk) Unclear/Unknown 

Low Risk Descriptions: 

Central allocation (telephone, web-based) 

Sequentially numbered, opaque and sealed envelopes 

High Risk Descriptions: 

Using an open random allocation schedule (list of random numbers) 

Assignment envelope were not opaque and/or sealed 

Alternation or Rotation 

Date of Birth 

Case Record Number 

Random Numbers Table 

Other explicitly unconcealed assignment procedure, specify 

x None 

Unclear/Unknown Description 

Inadequate information to make a judgement. 
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Blinding: Yes (Low Risk) X No (High Risk) Unclear/Unknown 


Low Risk Descriptions: 

No blinding, but reviewers determine there is little chance that that outcome 

and outcome measurements are influenced by lack of blinding. 

Participants and key study personnel blinding occurred and little chance 

blinding was broken. 

Some blinding occurred, but either participants or key personnel were not 

blinded and non-blinding had little chance of introducing bias. 

High Risk Descriptions: 

There is a lack of or incomplete blinding and there is a chance that outcome 

or outcome measurement will be influenced. 

There was an attempt to blind participant and key researchers but there is a 

good chance that blinding was broken. 

There was partial blinding and there is a good chance that this introduced 

bias, 
x None 

Unclear/Unknown Description 

Inadequate information to make a judgement. 

Study did not address outcome of interest. 

If blinded, check those who were blinded: 

Researcher Other (explain) 

Participant 

Intervener 

Assessor 

Employer 

Sequence Generation: 

Yes (Low Risk) No (High Risk) x Unclear/ Unknown 

Low Risk Descriptions: 

Random Numbers Table 

Random Number Generated 

Coin Flip 
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.Other, specify. 


High Risk Descriptions: 

Sequence generated by odd/ even birthdate 

Sequence generated by date of enrolment 

Sequence generated by CRN or other rule based method. Specify 

Assignment by participant preference 

Assignment by clinical judgement 

Assignment by pre-assessment tool 

Assignment by availability of the intervention 

None 

Unclear/Unknown Description 

x Inadequate information to make a judgement. 

Recruitment Pool: 

Referral Criterion Waiting List 

Existing Group Volunteer Not Reported 

_x Other, Explain: various hospitals and community rehabilitation facilities_ 

Comments: “recruited through various hospital and community rehabilitation facilities and 
centres in Hong Kong” (p. 1017) 

Number of Intervention Sites: One site: “Training took place at the computer laboratory at 
the Department of Rehabilitation Sciences at Hong Kong Polytechnic University” (Man et al., 
2013, p. 1019). 

If random sampling was used, was the same sampling procedure used at all sites? 

1. x Yes 2. No 3. Not used 

Source of Sample: 1. Public Agency 

2. Private Agency 

3. Not Reported 

4. x Other, explain: “various hospital and community rehabilitation 
facilities in Hong Kong” (Man et al., 2013, p. 1017) 
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Type of Comparison Group __x _Yes No 

If yes, check appropriate type 

Treatment as usual 

No intervention 

Alternate Service, Explain PEVTS “conventional training” 

Incomplete Outcome Data: 

x Y es (Low Risk) No (High Risk) Unclear/ Unknown 

Low Risk Descriptions 

No missing outcome data 

Equal amount of missing data across groups and all groups have similar 

justifications for missing data. 

x Dichotomous outcome data - ratio of proportion missing to observed is 
not relevant enough to bias the effect estimate. 

Continuous outcome data - not relevant enough to bias effect size estimate. 

Missing data have been appropriately imputed. 

High Risk Descriptions 

Unequal amount of missing data across groups and there is a good chance 

that this missing data is related to outcome across groups. 

Dichotomous data - the ratio of the proportion missing to observed is 

substantial enough to introduce bias. 

Continuous data - the amount of missing outcome data is large enough to 

introduce bias. 

Conducting an “As-treated” analysis with substantial discrepancy from 

random assignment. 

Inappropriate use of simple imputation. 

Unclear/Unknown Descriptions 

Information not reported. 

Incomplete reporting of attrition and exclusions such as no justification of 

outcome data. 

SUBJECT CHARACTERISITICS 


Gender Group l Group 2 

Female DNR % % 

Male DNR % % 
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Comments: Did not report 


Race/Ethnicity Group l 

African American DNR % 

Native American % 

Asian % 

Hispanic/Latino % 

White Non-Hispanic % 

Mixed % 

Other % 

Not Reported % 

Comments: Did not report 
Page: pages 1017 and 1021 


Group 2 

% 

% 

% 

% 

% 

% 

% 

% 


Education: 1. 

Some High School 

2. 

High School Graduate/GED 

3 - 

Some College 

4 - 

Graduated College 

Comments: Did not report 


Table 1: Group Characteristics 



Group 

Mean 

Age 

Males 

Pre- 

Post- 

Attrition 

Follow- 

F-Up 


Type 

Age 

Range 


test 

test 


Up 

Attrition 





% (n) 

n 

n 

n 

n 

n 


TX- 

DNR 

DNR 

DNR 

25 

20 

5 

20 


Group 

AIVTS 










C- 

DNR 

DNR 

DNR 

25 

20 

5 

20 


Group 

PEVTS 










Location: 1. X Urban 

2. Suburban 

3. Rural 

Comments: 
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Where Injury Occurred: 

Motor Vehicle/Traffic 

Fall 

Combat/T error 

Assault 

Colliding with moving or stationary object 

x Not Reported 
Time since Injury: Did not report 

Mean Age at Injury: Did not report, criteria states that participants must be between 18 and 
55 years of age 

Mean Time in Coma: LOC <6 hours; post-traumatic amnesia < 24 hours; aged 18 to 55; 
medically stable 

Severity of Injury: 1. Mild 

2. X Mild to Moderate (p. 1017) 

3. Moderate 

4. Not Reported 

5. Other, explain: 

Comments: 

Subject Population: 

x Civilian 
Military/Veteran 

Pre-Injury Employment: 1. Yes 2. No 3. X Unknown 

Comments: did not report 
Intervention Characteristics: 

Average length of intervention program: 12 session, each session was 20 to 25 minutes 

Length of time of participation activity 20 to 25 minutes 

Intervention Activities included _AIVST was a software program that was used to train 
specific vocation skills. PEVTS consisted of conventional psychoeducational vocational training. 
Both groups focused on specific vocational skills, practice in the specific skills, and review of 
new skills learned. 
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Type of Professional conducting intervention: vocational trainer 


Sample Status Assessed 

Only those that obtained full-time employed 

x Mixed - those that obtained full-time or part-time employed (p. 1021) 

Primary Type of Employment: 1. Retail Trade tx 1 (AIVTS) - 3; tx 2 (PEVTS) -1 

2. Transportation & Warehousing 

3. Information 

4. Finance 

5. Professional 

6. Education & health 

7. Leisure & Hospitality 

8. Construction 

9. Manufacturing 

10. Local government 

11. State Government 

12. Federal Government 

13. Other Service, explain: Clerical: treatment- 3; 

Labourer: tx 2 -1; Cleaner tx 2 - 1 

14. Not Reported 
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Comments: p. 1022 
Employment Outcome Measure(s): 

Employment outcome: 

Employed - dichotomous - 

a. Number employed 8 in treatment one and 4 in treatment two at 12 months 

b. Number unemployed, still 12 in treatment one and 16 in treatment two (p. 
1022) 


AIVT 

1 

3 

6 

PEVTS 

1 MO. 

3 

6 


MO. 

MOS. 

MOS. 



MOS. 

MOS 

Unemployed 

17 

14 

12 

Unemployed 

20 

16 

16 

Sheltered 

0 

0 

0 

Sheltered 

0 

0 

0 

Supported 

Employment 

2 

2 

3 

Supported 

Employment 

0 

1 

1 

Open/ Competitive 

1 

4 

5 

Open/ Competitive 

0 

3 

3 


Length of time employed: There is no information to say whether all maintained 
employment across time. One person obtained employment as a cleaner at one month and 
later obtained a job as a shop assistant. 

Hours worked per week: did not report 

Dropped Out Before Placement Occurred: did not report 

Comments: 

Selective Outcome Reporting 

Yes (Low Risk) x No (High Risk) Unclear/Unknown 

Low Risk Description 

Study protocol is available and all pre-specified outcomes that are of 

interest are reported in the pre-specified way. 

Study protocol is not available but all expected outcomes along with pre- 
specified outcomes are reported clearly. 

High Risk Description 
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Reporting of pre-specified outcomes is not complete. 

x Not all of pre-specified outcomes have been reported. Those that have 
been reported used a subset of data, used measured not pre-specified, or 
analysis that was not pre-specified. 

A t least one primary outcome was not pre-specified. 

A t least one outcome was not completely reported so that information can 
not be entered into meta-analysis. 

Study does not report findings for a key outcome that one would expect to 

have been reported for that particular type of study. 

Unclear/Unknown Description 

Inadequate information to make a judgement. 


Comments: Findings from one of the three outcome variables, Tower of London instrument, 
were not reported (p. 1021). 


Fidelity of Implementation 
Intervention implemented as described 

No 

Yes 

x Unclear 

Not Reported 

Comments: The assessment of fidelity implementation was not clear in the study. Under the 
second intervention group (PEVTS) training description, the authors mentions “fidelity of the 
manual with regard to clarity and relevance to problem solving was validated by an expert panel 
of four rehabilitation professionals... (p. 1020). It appears to be addressing only the manual used 
in the second intervention group. However, the treatment intervention consisted of progressing 
through a computer program, (p. 1020) 


Outcome Measures 


Outcome 

Measurement 

Reliability & 
Validity 

Type of Variable 

Employment 

Status 

Unemployed* 

Sheltered 


Ordinal* 
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Supported 

Open 




Comments: 

*We consider sheltered employment as unemployed and will be collapsed with unemployment. 
We consider supported employment as open/competitive employment and will collapse these 
two categories. We will then have a dichotomous variable response of unemployed/employed. 
Other primary outcomes not relevant to this systematic review: 

Wisconsin Card Sorting Test measures executive dysfunction, Tower of London Test measures 
planning and problem solving, Vocational Cognitive Rating Scale measures cognitive 
impairment at the workplace, self-efficacy, and comparison/contrast ability measurement. 

Mean hourly wage: Did not report 
Total days employed: Did not report 


Dichotomous Outcome Data 
Study: p. 1022 


Outcome 

Assessment 

Points 

Source 

Valid Ns 

Intervention 

One 

Intervention 

Two 

Statistics 

Page 

No. 

& 

Notes 

employment 

status 

Baseline 



3 

11 

10 

0 

ts 

11 

to 

0 

A 2 = 

11.14, P 

value at 

3 mos.- 

0.04 

P- 

1022 


Follow-Up 


50 % of N = 
40 in 

Intervention 

n = 20 at 

one, three, 
and six 

months 

n = 20 at one, 
three, and six 
months 

X 2 = 

8 . 00 , p 

value at 

6 mos. 

0.018 






Type 

Tx l/AIVTS 

Type tx 2 
/PEVTS 

Log rank 
ratio 
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OR 








95% Cl 








Dfl 








Chi- 

Square 

3 mos- 

11.14 

6 mos. 

8.00 



AIVT 

1 

MO. 

3 

MOS. 

6 

MOS. 

PEVTS 

1 MO. 

3 

MOS. 

6 

MOS 

Unemployed 

17 

14 

12 

Unemployed 

20 

16 

16 

sheltered 

0 

0 

0 

sheltered 

0 

0 

0 

Supported 

Employment 

2 

2 

3 

Supported 

Employment 

0 

1 

1 

Open/ Competitive 

1 

4 

5 

Open/ Competitive 

0 

3 

3 


Continuous Outcome Data 

Study: no other relevant data was reported. 


Outcome 

Assessment 

Points 

Source 

Valid 

N 

Intervention 

One 

Intervention 

Two 

Statistics 

Page 

No. & 

Notes 


Baseline 



n 

n 

p value 



Follow-Up 



n 

n 

t 






Means 

Means 

F 
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SDs 

SDs 

df 






Type 

Type 

ES 








Other 








Covariates 



14-4- Appendix 4: Coding Form for Primary Research Characteristics for Employment 
Interventions for Return to Work in Working-Aged Adults Following Traumatic Brain 
Injury (TBI) Systematic Review 

Author (s): 

Salazar, A.M., Warden, D. L., Schwab, K. A., Braverman, S. E., Walter, J., Cole, R., Rosner, M. 
M., Martin, E. M., Ecklund, J., Ellenbogen, R. G., (2000). Cognitive rehabilitation for traumatic 
brain injury: A randomized trial. JAMA , 283, 3075-3081. 

Other Articles published on this study: 

Braverman, S. E., Spector, J., Warden, D. L., Wilson, B. C., Ellis, T. E., Bamdad, M. J., & Salazar, 
A. M. (1999). A multidisciplinary TBI inpatient rehabilitation programme for active duty service 
members as part of a randomized clinical trial. Brain Injury, 13, 405-415. 

Warden, D. L., Salazar, A. M., Martin, E. M., Schwab, K. A., Coyle, M., Walter, J. &the DVHIP 
Study Group. (2000). A home program of rehabilitation for moderately severe traumatic brain 
injury Patients. Journal of Head Trauma Rehabilitation, 15, 1092-1102. 

Source: 1. Journal - JAMA, 283, 3075-3081 

2. Book or Book Chapter 

3. Master Thesis/Doctoral Dissertation 

4. Presentation 
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5. Government Agency Report 

6. Organization Report 
Intervention: 

Employment Intervention: l. Yes X 2. No (will be excluded from study) 

Participants Had TBI: l. Yes 2. No (will be excluded from study) 

Comments: _Pre 9/ 11 military sample; non-combat related TBI; moderate to severe TBI; no 
prior severe TBI; TBI occurred within 3 months of randomization; all had recovered sufficiently 
from TBI to participate in study 
Alcohol related - 31% 

Assault related - 22% 

Type of Intervention: transitional program 

According to Salazar, after an extensive assessment, participants received both group and 
individual treatment, which was conducted by a physiatrist and assisted by occupational 
therapist, speech pathologist, and two rehabilitation assistants (Salazar et al., 2000). If a 
participant needed physical therapy, neurological, or psychiatric consults, these were provided. 
Participants were on a minimum care ward and were required to “conform to military 
standards” (p. 3076). Their time was structured and included physical fitness, group and 
individual cognitive treatment, speech therapy, occupational therapy, and coping skills therapy. 
Participants received assistants in community re-entry and work therapy. (Salazar et al., 2000, 

pages 3076-3077) 

Design: 

Randomized Controlled Trials (RCTs) _RCT with parallel intervention groups 

Quasi-Experimental 

Single Subject Experimental 

Other (will be excluded from study) 

Sampling: 

Random, specify type 

x Other, explain convenience - all were injured and seeking treatment at Walter Reed Army 
Medical Center 


Allocation Concealment Yes (Low Risk) _ No (High Risk) x Unclear/Unknown 
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Low Risk Descriptions: 

Central allocation (telephone, web-based) 

Sequentially numbered, opaque and sealed envelopes 

High Risk Descriptions: 

Using an open random allocation schedule (list of random numbers) 

Assignment envelope were not opaque and/or sealed 

Alternation or Rotation 

Date of Birth 

Case Record Number 

Random Numbers Table 

Other explicitly unconcealed assignment procedure, specify 

None 

Unclear/Unknown Description 

x Inadequate information to make a judgement. 

Blinding: x Yes (Low Risk) No (High Risk) Unclear/Unknown 

Low Risk Descriptions: 

No blinding, but reviewers determine there is little chance that that outcome 

and outcome measurements are influenced by lack of blinding. 

Participants and key study personnel blinding occurred and little chance 

blinding was broken. 

x Some blinding occurred, but either participants or key personnel were not 
blinded and non-blinding had little chance of introducing bias. 

High Risk Descriptions: 

There is a lack of or incomplete blinding and there is a chance that outcome 
or outcome measurement will be influenced. 

There was an attempt to blind participant and key researchers but there is a 

good chance that blinding was broken. 

There was partial blinding and there is a good chance that this introduced 

bias. 

None 

Unclear/Unknown Description 

Inadequate information to make a judgement. 

Study did not address outcome of interest. 
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If blinded, check those who were blinded: 

x Researcher Other (explain) 

Participant 

x Intervener 

Assessor 

Employer 

Comment: Outside statistician randomized subjects. Salazar (2000) stated that statistician 
used variable sized blocks in an effort to prevent researchers from determining the 
assignments. Each treatment was implemented by separate teams. These teams of staff did 
not work with each other or the assessment team. P. 3076 

Sequence Generation: 

x_ Yes (Low Risk) No (High Risk) Unclear /Unknown 

Low Risk Descriptions: 

Random Numbers Table 

x Random Number Generated 
Coin Flip 

Other, specify 

High Risk Descriptions: 

Sequence generated by odd/ even birthdate 

Sequence generated by date of enrolment 

Sequence generated by CRN or other rule based method. Specify 

Assignment by participant preference 

Assignment by clinical judgement 

Assignment by pre-assessment tool 

Assignment by availability of the intervention 

None 

Unclear/Unknown Description 

Inadequate information to make a judgement. 

Recruitment Pool: 

Referral _x_ Criterion Waiting List 

Existing Group Volunteer Not Reported 
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Other, Explain 

Comments: Blocked Randomization was used. Size of blocks varied. 

Number of Intervention Sites: l 

If random sampling use, was the same sampling procedure used at all sites? Yes No 

Source of Sample 

5. Public Agency 

6. Private Agency 

7. Not Reported 

8. X_ Other (explain) Military Hospital 

Type of Control Group 

Treatment as usual 

Wait List 

X Alternate Service, Explain _Home based less intensive -see below for more information 

Unclear 

Not reported 

According to Warden and associates, participants in the Home based program, which was 
considered a “low intensity, low cost treatment” was a comparison group (Warden et al., 1999, p. 
1095). This program aimed at having the participant gradually resume mental and physical 
activities. After an assessment, a psychiatric nurse with master’s level training first met with the 
participant at his/her home and discussed participants’ prior treatment and explained the home 
based program intervention. This nurse called weekly at a mutually agreed upon time. The 
therapeutic techniques used during the weekly telephone calls consisted of developing a 
therapeutic alliance, fostering self-direction, supporting coping in the adjustment to TBI, and 
assisting patient and family in problem solving. There were also participant instructions: read at 
least 30 minutes a day, play games that used reasoning skills (e.g., Pictionary, card games, 
computer games), and engaged in 30 minutes of physical exercise at own pace per day. Other 
recommended strategies in which the participants were recommended to engage included: use 
highlighters when reading, use index cards when reading to block the words below the reading 
line, eliminate noisy distractions, establish daily routine, restrain from drinking alcohol, use 
weekly pill boxes, use to-do checklists, use calendar for appointments, and create and use a 
shopping list (Warden et al., 2000, p ages 1094 to 1095). 

Incomplete Outcome Data: 

x Yes (Low Risk) No (High Risk) Unclear/Unknown 
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Low Risk Descriptions 

No missing outcome data 

Equal amount of missing data across groups and all groups have similar 

justifications for missing data. 

Incomplete Outcome Data: 

_x_ Yes (Low Risk) No (High Risk) Unclear/ Unknown 

Low Risk Descriptions 

No missing outcome data 

Equal amount of missing data across groups and all groups have similar 

justifications for missing data. 

x Dichotomous outcome data - ratio of proportion missing to observed is 
not relevant enough to bias the effect estimate. 

Continuous outcome data - not relevant enough to bias effect size estimate. 

Missing data have been appropriately imputed. 

High Risk Descriptions 

Unequal amount of missing data across groups and there is a good chance 

that this missing data is related to outcome across groups. 

Dichotomous data - the ratio of the proportion missing to observed is 

substantial enough to introduce bias. 

Continuous data - the amount of missing outcome data is large enough to 

introduce bias. 

Conducting an “As-treated” analysis with substantial discrepancy from 

random assignment. 

Inappropriate use of simple imputation. 

Unclear/Unknown Descriptions 

Information not reported. 

Incomplete reporting of attrition and exclusions such as no justification of 

outcome data. 

Dichotomous outcome data - ratio of proportion missing to observed is 

not relevant enough to bias the effect estimate. 

Continuous outcome data - not relevant enough to bias effect size estimate. 

Missing data have been appropriately imputed. 

SUBJECT CHARACTERISITICS 
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SES _Consisted both enlisted and commissioned officers; Hospital group: 72% Jr enlisted, 21 
non-commissioned officer, and 7% commissioned officers (Braverman et al., 1999, p. 407); 
Home group: 30% of home group were sergeants or higher (Salazar et al., 2000; Warden et al., 
2000). The Salazar and Warden articles did not elaborate any further on the rank of participants 
as the Braverman article did on the hospital group. 


Gender 

Group 1 

Group 2 


Hosp. 

Home 

Female 

3 % 

4 % 

Male 

93 % 

96 % 

Race/Ethnicity 

Group 1 

Group 2 

White 

6q% 

70% 

African American 

24 % 

% 

other 

7 % 

% 


% 

% 


% 

% 


% 

% 


% 

% 


Comments: Did not report other races/ethnicity 


Education Hosp Home 

Some High School 

High School Graduate/ GED 61% 44% of Home group had “some college or 

more” (Warden, 2000, p. 1098) 

Some College 28% 

Graduated College 11% 

Comments: The hospital group information came from Braverman et al., 1999 and was more 
detailed. Salazar et al (2000) and Warden et al. (2000) addressed both groups but did not 
provided detailed information on the demographics. 


Table 1: Group Characteristics: 



Group 

Type 

Mean 

Age 

Age 

Range 

Males 

% (n) 

Pre- 

test 

n 

Post 

test 

n 

Attrition 

n 

Follow- 

Up 

n 

F-UP 

Attritio 

n 

n 

Group 

Hosp 

25 

(6.63) 


93 

(62) 

67 

60 

7 

60 

0 
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Group 

Home 

26 

(6.22) 


96 

(51) 

53 

47 

6 

47 

0 

Group 










Group 











Location of Treatment 
x Urban Rural 

Suburban Other (explain) 

Comments: 

Where Injury Occurred: 

x Motor Vehicle/Traffic 
Fall 

Combat/T error 

x Assault 

Colliding with moving or stationary object 

_x_ Other: “Alcohol Related Accidents” (Braverman et al., p. 406) 

Not Reported 

Time Since Injury: In hospital: M = 38 (SD=23.6) In-Home: M = 39 (SD=33.2) 
Mean Age at Injury: Did not report 

Mean Time in Coma: In-hospital: > 1 hr. = 53%, > 24 hr. = 30%; 

In-home: >1 hr. = 76%, >24hr 38% 

Severity of Injury 

Mild Comments: 

Mild to Moderate 

Moderate 

Not Reported 

x Other (explain) Moderate to severe 

Subject Population: 

Civilian 


116 


The Campbell Collaboration | www.campbellcollaboration.org 





117 


x Military/Veteran 

Pre-Injury Employment _A 11 Military non-combat related 
Intervention Characteristics: 

Average length of intervention program 8 weeks for both groups 

Length of time of participation activity: home 30 min. a day; hospital: 2 hours of work 
therapy 

Number of Sessions: 8 weeks - 5 days a week = 40 sessions 

Intervention Activities: The hospital group participants were in the transitional program. It 
included a residential treatment program using an interdisciplinary program, which included 
cognitive rehab treatment, Prigatano model of milieu therapy, and Ben-Yishay model of 
incorporating goal-oriented employment type work tasks for several hours each day. An 
occupational therapist conducted the work therapy and included “work performance assessment 
before return to a competitive work environment, recommendations on work potential, and 
remediation of the deficits in work skills” (Braverman et al, 1999, p. 413). 

The home-based treatment group received weekly phone calls from a psychiatric nurse. The 
nurse developed a therapeutic alliance, fostered self-direction, supported coping in the 
adjustment to TBI, and assisted patient and family in problem solving. The nurse gave each 
participant the following instructions: read at least 30 minutes a day, play games that used 
reasoning skills (e.g., Pictionary, card games, computer games), engage in 30 minutes of 
physical exercise at own pace per day. Other recommended strategies in which the participants 
were recommended to engage included: use highlighters when reading, use index cards when 
reading to block the words below the reading line, eliminate noisy distractions, establish daily 
routine, restrain from drinking alcohol, use weekly pill boxes, use to-do checklists, use calendar 
for appointments, and create and use a shopping list. (Warden et al., 2000, p ages 1094 to 

1095). 

Type of Professional conducting intervention - Hosp group - occupational therapist and 
rehabilitation assistant 

Sample Status Assessed 

Only those that obtained full-time employed 

Mixed - those that obtained full-time or part-time employed 

x Other: Did not report whether full-time or part-time employed 
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Primary Type of Employment: were related to each participant’s military occupational 
speciality (MOS); 

Retail Trade 

Transportation & Warehousing 

Information 

Finance 

Professional 

Education & health 

Leisure & Hospitality 

Construction 

Manufacturing 

Local government 

State government 

Federal government 

x Other Service: 
x Not Reported 
Comments: 

Of those employed at 12 months, 22% were employed outside of the military. (Salazar et al., 
200, p. 3079) The types of civilian jobs were not specified. 

Employment Outcome Measure(s): 

Employment outcome: 

Employed - dichotomous - 

Number employed: @ 12 mos. - hospital 60/67 (90%); Home 50/53 (94%) 
Number unemployed, still 

Length of Time to Place in Employment: Not Reported 

Length of Time Employed: NR 

Hours worked per week: NR 

Selective Outcome Reporting 

Yes (Low Risk) x No (High Risk) Unclear/Unknown 

Low Risk Description 
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Study protocol is available and all pre-specified outcomes that are of 

interest are reported in the pre-specified way. 

Study protocol is not available but all expected outcomes along with pre- 
specified outcomes are reported clearly. 

High Risk Description 

x Reporting of pre-specified outcomes is not complete. 

Not all of pre-specified outcomes have been reported. Those that have 

been reported used a subset of data, used measured not pre-specified, or 
analysis that was not pre-specified. 

A t least one primary outcome was not pre-specified. 

A t least one outcome was not completely reported so that information can 
not be entered into meta-analysis. 

Study does not report findings for a key outcome that one would expect to 

have been reported for that particular type of study. 

Unclear/Unknown Description 
Inadequate information to make a judgement. 


Comments: “Patients were re-evaluated at 6, 12, and 24 months with the multidisciplinary 
baseline test and additional psychosocial outcome measures, including return to work” 
(Salazar et al., 2000, p. 3077). Only 12 months were reported and not 6 months. 

Fidelity of Implementation 
Intervention implemented as described 
x No 

Yes 

Unclear 

Not Reported 

Comments: Fidelity of implementation was not mentioned in any of the articles on this project. 
There was no indication of fidelity being addressed except for a mention of a checklist to 
determine what the participants were doing, that nurses in the home program used during 


119 


The Campbell Collaboration | www.campbellcollaboration.org 



120 


telephone calls (Warden, 2000, p. 1096). Even this would not meet the criteria of assessing 
fidelity. There was no indication of any assessment in the hospital group. 
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Coding Form for Primary Research Characteristics for Employment Interventions for Return to 
Work in Working-Aged Adults Following Traumatic Brain Injury (TBI) Systematic Review 


BASIC INFORMATION 
Author(s): 

1. Twamley, E. W., Thomas, K. R., Gregory, A. M., Jak, A. J., Bondi, M. W., Delis, D. C., & 
Lohr, J. B. (2014). CogSMART compensatory cognitive training for traumatic brain 
injury: Effects over 1 year. Journal of Head Trauma Rehabilitation, 
...DOI:io.io 97 /HTR.ooooooooooooooo 76 

2. Twamley, E. W., Jak, A. J., Dean, C. D., Bondi, M. W., Lohr, J. B. (2015). Cognitive 
symptom management and rehabilitation therapy (CogSMART) for veterans with 
traumatic brain injury: Pilot randomized controlled trial, JRRD, 51(1), 59-70. (primary 
employment outcome study) 

3. Walter, K. H., Jak, A. J., & Twamley, E. W. (2015). Psychiatric comorbidity effects on 
compensatory cognitive training outcomes for veterans with traumatic brain injuries. 
Rehabilitation Psychology, Advance online publication, July 6, 2015. 
http://dx.doi.ora/io.i 027 /repooooo 4 Q 


Date of Pub: 2014 
Source: 1. X Journal 

2. Book or Book Chapter 

3. Master Thesis/Doctoral Dissertation 

4. Presentation 

5. Government Agency Report 

6. Organization Report 

Intervention: 

Employment Intervention: 1. X Yes 2. No (will be excluded from study) 

Participants Had TBI: 1. X Yes 2. No (will be excluded from study) 

Comments: 

Treatment group received 12 weeks (3 mos.) of Cognitive Symptom Management and 
Rehabilitation Therapy (CogSMART) and 12 months of supported employment for one hour 
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from employment specialist. The second intervention group received three months of enhanced 
supported employment (ESE) by their employment specialist. 

Type of Intervention: cognitive and supported employment 

Provider: Demand Side _x_ Supply Side 

Duration of Intervention: l year 

Design: 

Randomized Controlled Trials (RCTs): RCT with parallel treatment groups 

Quasi-Experimental 

Other (will be excluded from study) 

Sampling: 

Random, specify type 

x Other, explain referrals 

Allocation Concealment Yes (Low Risk) No (High Risk) x Unclear/Unknown 

Low Risk Descriptions: 

Central allocation (telephone, web-based) 

Sequentially numbered, opaque and sealed envelopes 

High Risk Descriptions: 

Using an open random allocation schedule (list of random numbers) 

Assignment envelope were not opaque and/or sealed 

Alternation or Rotation 

Date of Birth 

Case Record Number 

Random Numbers Table 

Other explicitly unconcealed assignment procedure, specify 

None 

Unclear/Unknown Description 

x Inadequate information to make a judgment. 

Blinding: Yes (Low Risk) x No (High Risk) Unclear/Unknown 

Low Risk Descriptions: 

No blinding, but reviewers determine there is little chance that that outcome 

and outcome measurements are influenced by lack of blinding. 

Participants and key study personnel blinding occurred and little chance 

blinding was broken. 

Some blinding occurred, but either participants or key personnel were not 

blinded and non-blinding had little chance of introducing bias. 

High Risk Descriptions: 

x There is a lack of or incomplete blinding and there is a chance that outcome 
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or outcome measurement will be influenced. 

There was an attempt to blind participant and key researchers but there is a 

good chance that blinding was broken. 

There was partial blinding and there is a good chance that this introduced 

bias. 

None 

Unclear/Unknown Description 

Inadequate information to make a judgement. 

Study did not address outcome of interest. 

If blinded, check those who were blinded: 

Researcher Other (explain) research assistant 

Participant 

Intervener 

Assessor 

Employer 

Comment: “The research assistant was blind to initial randomization assignment at the 
baseline assessment but was not blind to the assigned treatment group for subsequent 
assessments” (Walter, Jak, &Twamley, 2015, p. 2) 

Sequence Generation: 

x Yes (Low Risk) No (High Risk) Unclear/Unknown 

Low Risk Descriptions: 

Random Numbers Table 

x Random Number Generated - used Randomization.com 
Coin Flip 

Other, specify 

High Risk Descriptions: 

Sequence generated by odd/ even birthdate 

Sequence generated by date of enrolment 

Sequence generated by CRN or other rule based method. Specify 

Assignment by participant preference 

Assignment by clinical judgement 

Assignment by pre- assessment tool 

Assignment by availability of the intervention 

None 

Unclear/Unknown Description 

Inadequate information to make a judgement. 

Recruitment Pool: 

x Referral Criterion Waiting List 
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Existing Group Volunteer Not Reported 

Other, Explain 

Comments: 

Number of Intervention Sites: The number of interventions sites was not reported in any of 
the three articles (Twamley, 2014; Twamley, 2015, Walter, Jak & Twamley, 2015). However, 
Twamley (2014) reported the location of the supported intervention in the following manner: 
“Consistent with the supported employment model, services were offered at locations of the 
participant’s choosing (e.g., career center, home, coffee shop, library, or VA clinic)” (Twamley, 
2014, p. 62). It must be noted here that supported employment intervention occurs at a number 
of locations including assistance at the work site during employment. The authors are concerned 
that true supported employment was not provided during this intervention. 

If random sampling was used, was the same sampling procedure used at all sites? 

1. Yes 2. No 3. x Not used 

Source of Sample: 1. _X Public Agency — VA San Diego Health System Wellness and 

Vocational Enrichment Clinic, BBI Cognitive Rehabilitation Clinic, Poly trauma Clinic, and 
Neuropsychological Assessment Unit (Twamley et al., 2014a, p. 61) 

2. Private Agency 

3. Not Reported 

Comments: “Several clinics within the VA San Diego Health Care System” were used to recruit 
participants (Walter, Jak, & Twamley, 2015, p. 2). 

Type of Comparison Group x Yes No 

If yes, check appropriate type 

Treatment as usual 

No intervention 

x Alternate Service, Explain: Used another intervention - enhanced supported employment 
Incomplete Outcome Data: 

x Yes (Low Risk) No (High Risk) Unclear/ Unknown 

Low Risk Descriptions 

No missing outcome data 

Equal amount of missing data across groups and all groups have similar 

justifications for missing data. 

x Dichotomous outcome data - ratio of proportion missing to observed is 
not relevant enough to bias the effect estimate. 

Continuous outcome data - not relevant enough to bias effect size estimate. 

Missing data have been appropriately imputed. 

High Risk Descriptions 
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Unequal amount of missing data across groups and there is a good chance 

that this missing data is related to outcome across groups. 

Dichotomous data - the ratio of the proportion missing to observed is 

substantial enough to introduce bias. 

Continuous data - the amount of missing outcome data is large enough to 

introduce bias. 

Conducting an “As-treated” analysis with substantial discrepancy from 

random assignment. 

Inappropriate use of simple imputation. 

Unclear/Unknown Descriptions 
Information not reported. 

Incomplete reporting of attrition and exclusions such as no justification of 

outcome data. 

5. Alternate Service, explain: 

6. Unclear 

7. Not reported 

SUBJECT CHARACTERISITICS 
Page: Twamley et al., 2014, p. 4 

Gender Group 1 Group 2 

Female 4% (n=i) 4% (n=i) 

Male 96% (11=24) 96% (n=24) 

Comments: 

Race/Ethnicity Group 1 Group 2 

African American % % 

Native American % % 

Asian % % 

Hispanic/Latino 36% 36% 

White Non-Hispanic 24% 48% 

Mixed % % 

Other % % 

Not Reported % % 

Comments: There were differences in the reported ethnic characteristics of both groups over 
the documents. 

Education: 1. Some High School 

2. x_ High School Graduate/GED - Treatment group mean 12.88 (SD = 1.27) 
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3. x_ Some College - second intervention group mean 14.00 (SD=i.8o) 
(Twamley et al., 2015 

P- 4 ) 

4. Graduated College 

Comments: did not give percentage falling into each educational level 


Table 1: Group Characteristics (data in table below from Twamley et al., 2015). 



Group 

Type 

Mean 

Age 

Age 

Rang 

e 

Males 

96 (n) 

Pre- 

test 

n 

Pos 

t- 

test 

n 

Attrition 

n 

Follow 

-Up 

n 

F-UP 

Attrition 

n 

Group 

TX 

29.68 


96%(24) 

25 

23 

2 

21 

2 

Group 

C 

33-84 


96%(24) 

25 

24 

1 

21 

3 










Total 

Attrition 

4 in each 

group 


Location of Treatment 
Urban Rural 

Suburban Other (explain) 

Comments: 

Where Injury Occurred: did not report (DNR) 

Where Injury Occurred: 

Motor Vehicle/Traffic 

Fall 

Combat/T error 

Assault 

Colliding with moving or stationary object 

x Not Reported 

Time since Injury: CogSMART Plus: M = 3.6 years (SD = 2.7) 

Enhanced Supported Employment: 5.1 years (SD = 5.9) 
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Mean Age at Injury: did not report 

Mean Time in Coma: CogSMART Plus: Median = 6.5 minutes, Range = 0-420 

Enhanced Supported Employment: Median = 1.0 minutes 
Range = o - 1,440 minutes 

Severity of Injury: 1. Mild 

2. x Mild to Moderate (Twamley et al., 2015) 

3. Moderate 

4. Not Reported 

5. Other, explain: 

Subject Population: 

Civilian 

x Military/Veteran 

Pre-Injury Employment: 1. x Yes 2. No 3. Unknown 

Comments: All participants were veterans of OEF/OIF (Twamley et al., 2015, p. 1). 
Intervention Characteristics: 

Average length of intervention program: CogSMART group received 12 weeks of 
Cognitive Symptom Management and 12 months of supported employment. Enhanced 
Supported Employment received three months of enhanced supported employment and nine 
more months of supported employment. 

Length of time of participation activity: CogSMART: 1 hour per week in CogSMART plus 2 
visits of supported employment per week. Enhanced Supported Employment: 2 visits per week 
for first three months and then one visit per week for the rest of the year. 

Number of Sessions: CogSMART: 3 sessions per week. Enhanced Supported Employment: 2 
session per week for first three months and then one visit per week for the rest of the year. 

Intervention Activities included: The treatment group received three months of 
CogSMART, which addressed post-concussive symptoms, prospective memory, attention and 
vigilance, learning and memory, and executive functioning (see cogsmart.com for more 
information). The treatment group received one hour of CogSMART and one hour of supported 
employment per week for the first three months and then one hour of supported employment 
for the rest of the year. The second intervention group received two hours of supported 
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employment for the first three months and then one hour of supported employment for the rest 
of the year. Separate employment specialists worked with each group. 

Type of Professional conducting intervention: Supported Employment Specialists 
Sample Status Assessed 

Only those that obtained full-time employed 

Mixed - those that obtained full-time or part-time employed 

x Did not Report whether it was all full-time or a mixture of full-time and part-time 

Primary Type of Employment: l. Retail Trade 

2. Transportation & Warehousing 

3. Information 

4. Finance 

5. Professional 

6. Education & health 

7. Leisure & Hospitality 

8. Construction 

9. Manufacturing 

10. Local government 

11. State Government 

12. Federal Government 

13. Other Service, explain: 

14. x Not Reported 

Employment Outcomes Measure(s): 

a. Number employed: 13/25 CogSMART Plus and 13/25 Enhanced Supported 
Employment 

b. Number unemployed, still: 12 in CogSMART Plus and 12 in Enhanced Supported 
Employment (Twamley et al., 2015, p. 8) at 12 months 

Length of time employed: not reported 
Hours worked per week: not reported 

Number who dropped out before placement occurred: 10 dropped out of study, 5 from 
each group (Twamley et al., 2015, p. 8). Two decided not to pursue work, 1 moved, 5 lost to 
follow-up (Twamley et al., 2015, p. 3). 

Comments: 

“Post-treatment data were available for 34 participants at 3 months (16 in supported 
employment plus CogSMART [3 with modTBI] and 18 in enhanced supported employment [4 
with modTBI]” (Twamley 2015, p. 62). 
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“...15 of the 16 participants randomized to receive supported employment plus CogSMART 
completed all 12 sessions and 1 participant completed 8 sessions” (Twamley 2015, p. 62). 
Selective Outcome Reporting 

Yes (Low Risk) x No (High Risk) Unclear/Unknown 

Low Risk Description 

Study protocol is available and all pre-specified outcomes that are of 

interest are reported in the pre-specified way. 

Study protocol is not available but all expected outcomes along with pre- 
specified outcomes are reported clearly. 

High Risk Description 

x Reporting of pre-specified outcomes is not complete. 

Not all of pre-specified outcomes have been reported. Those that have 

been reported used a subset of data, used measured not pre-specified, or 
analysis that was not pre-specified. 

A t least one primary outcome was not pre-specified. 

A t least one outcome was not completely reported so that information can 
not be entered into meta-analysis. 

Study does not report findings for a key outcome that one would expect to 

have been reported for that particular type of study. 

Unclear/Unknown Description 

Inadequate information to make a judgment. 

Comments: Twamley et al. (2015) stated, “This is the first report to include the 6-month or 12 

month outcomes” (p. 6). The 6-month employment status data were not reported. 

Fidelity of Implementation 
Intervention implemented as described 

No 

x Yes 

Unclear 

Not Reported 

Comments: CogSMART is manualized. “CogSMART sessions were recorded and 20% of 
recordings were randomly selected for fidelity ratings to ensure that adherence to the treatment 
manual was maintained” (Walter, Jak, & Twamley, 2015, p. 2). 


129 


The Campbell Collaboration | www.campbellcollaboration.org 



“All assessments were administered by a trained research assistant who followed standardized 
procedures and achieved high inter rater reliability (i.e., rs= .90)” (Walter, Jak, & Twamley, 
2015, p- 2). 


Outcome Measures 


Outcome 

Measurement 

Reliability & 

Validity 

Type of 
Variable 

Attained Competitive 
Employment Within 14 
Weeks 

Yes/No 

None 

Dichotomous 






Mean hourly wage: did not report 
Total days employed: did not report 

Comment: Primary outcome variables for this study included: IQ using Wide Range 
Achievement test (WRAT3), prospective memory with Memory for Intentions Screening Test 
(MIST), attention and working memory using Wechsler Adult intelligence Scale (3 rd ed), verbal 
and memory using CVLT-II, and executive functioning using Delis-Kaplan Executive Function 
System (D-KEFS), post-concussive symptom severity using Neurobehavioral Symptom 
Inventory (NSI), PTSD symptoms using Clinician Administered PTSD Scale (CAPS), depressive 
symptom severity with Hamilton Depression Rating Scale (HAM-D), global quality of life using 
quality of life interview brief version 

“Data regarding job attainment, hours worked, and wages earned were collected weekly” 
(Twamley 2015, p. 64) but were not reported in either article. 
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Dichotomous Outcome Data 
Study: Twamley et al., 2015 


Outcome 

Assessment 

Points 

Source 

Valid Ns 

Intervention 

One 

Intervention 

Two 

Statistics 

Page No. 

& Notes 

Employment 

status 

Baseline 

0% 

All 

unemployed 



n at 12 mos. 

=21 

n at 12 mos. 

=21 

X 2 =o 
p value 

at 12 

months 

1.00 

More 

subjects 

were 

included 

in the 12 

month 

report 

than in 
the 3 
month 

report. 


Follow-Up 

0% 

All 

unemployed 


50% of N = 

40 in 

Intervention 

n = 16 at 3 

mos. 

n = 18 at 3 

mos. 

X 2 =2.1 
p value 
at 3 

months 

was 

0.49 






Type 

Cognitive + 

Supported 

Employment 

Type 

Supported 

Employment 



3 Month 

Data 

txi 

tx 2 

12 Month 

Data 

tx 1 

tx 2 



Employed 

50 % 

26% 

Employed 

65% or 

13/20 

65% or 

13/20 

95% Cl 


Not 

Employed 

50 % 

74 % 

Not 

Employed 

35% or 7/20 

35% or 7/20 

df 

1 







Chi- 

Square 

2.10 
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Effect 0.49 at 

size 3 

months 


Continuous Outcome Data — no relevant continuous data were reported. 
Study: 


Outcome 

Assessment 

Points 

Source 

Valid 

N 

Intervention 

One 

Intervention 

Two 

Statistics 

Page 

No. & 

Notes 


Baseline 



n 

n 

p value 



Follow-Up 



n 

n 

t 






Means 

Means 

F 






SDs 

SDs 

df 






Type 

Type 

ES 








Other 








Covariates 
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* For supported employment that states a length of intervention duration, we will use this 
information. For those supported employment studies in which the supportive intervention is 
ongoing, we will use the data collection times, such as 12 months and 30 months in the 1992 
Ellerd & Moore study. 
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P.0. Box 4004, Nydalen 

Pilestredet Park 7 
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Phone: (+47) 23 25 50 00 

N-0403 Oslo, Norway 

(Entrance from Stensberggata) 




■ CampbellCollaboration 


About this review 

After sustaining a traumatic brain injury (TBI), people often struggle to obtain competitive 
employment and unemployment among individuals with TBI is considerably higher than for 
individuals without disabilities. This review examines the effectiveness of vocational rehabil- 
itation (VR) interventions to help adults with TBI attain competitive employment. 
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